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VOLUME  I 


DEPARTMENT  OF  DEFENSE 
Washington,  D.  C. 


OBJECTIVES  OF  THE  CONFERENCE 


1.  Analyze  the  principal  quality  and  reliability  problems  that 
confront  the  Department  of  Defense. 

2.  Assess  pertinent  current  efforts  and  proposed  plans  which 
are  aimed  at  solving  these  problems. 

3.  Exchange  ideas  on  solutions  of  these  problems. 

4.  Recommend  to  executive  levels  of  the  Office  of  the  Secretary 
of  Defense,  the  military  departments  and  the  Defense  Supply 
Agency:  (a)  appropriate  and  immediate  actions  to  resolve 
specific  quality  and  reliability  problems,  and  (b)  long-range 
plans  and  programs  for  improving  quality  and  reliability 
assurance  efforts. 
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ASSISTANT  SECRETARY  OF  DEFB4SE 
WASHINGTON,  D.C,  30*01 


INSTALLATIONS  AND  LOWSTKS 


MEMORANDUM  FOR  THE  CONFEREES 


Welcome  to  the  first  Defense  Conference  on  Quality  and  Reliability  Management. 

We  are  confident  that  this  conference  will  bring  into  focus  the  major  problems 
that  confront  all  components  of  the  Department  of  Defense  in  their  efforts  to 
improve  quality  and  reliability  assurance.  The  purpose  of  this  conference  is 
to  provide  a  forum  for  interchange  of  ideas  among  knowledgeable  personnel  of 
the  Services  and  the  Defense  Supply  Agency  which,  hopefully,  will  lead  toward 
the  solution  of  many  of  these  problems. 

The  increasing  complexity  of  our  military  materiel  has  caused  a  corresponding 
complexity  in  the  tasks  associated  with  the  prediction,  achievement,  maintenance, 
and  assessment  of  the  necessary  quality  and  reliability.  As  the  costs  of 
materiel  have  increased,  so  have  the  costs  of  quality  and  reliability  assurance. 
Improved  management  of  quality  and  reliability  will  contribute  significantly 
toward  advancing  our  technology,  decreasing  acquisition  lead  time,  conserving 
technical  resources  and  improving  logistical  support  of  our  weapon  systems 
and  equipment.  Better  management  in  all  of  these  areas  will  also  result  in 
considerable  cost  benefits  on  behalf  of  industry  and  Government. 

It  is  both  essential  and  timely  that  we  examine  our  policies,  procedures  and 
techniques  in  order  to  improve  our  quality  and  reliability  assurance  efforts 
throughout  the  Defense  community.  You  have  been  carefully  selected  as 
Conference  participants  in  recognition  of  your  ability  to  contribute  toward 
this  end.  We  are  expectant,  therefore,  that  the  first  Defense  Conference 
on  Quality  and  Reliability  Management  will  prove  to  be  highly  productive. 

We  look  forward  with  great  interest  to  the  results  of  your  efforts. 


Assistant  Secretary  of  Defense  Deputy  Director,  Defense 

(Installations  and  Logistics)  Research  and  Engineering 
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Deputy  Director  of  Defence  Research  and  Engineering  Finn  J.  Larsen 
(Lei:)  a?td  Conference  Steering  Committee  Chairman  John  J.  Riordan 
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FOREWORD 


These  Proceedings  are  a  record  of  the  First  Department  of 
Defense  Conference  on  Quality  and  Reliability  Management.  The 
Conference  brought  together  more  than  130  participants  from 
Department  of  Defense  activities  National  Aeronautics  and  Space 
Administration,  Federal  Aviation  Agency,  General  Services 
Administration,  and  National  Bureau  of  Standards.  More  than  100 
guests  were  present  from  the  Atomic  Energy  Commission,  Food  and 
Drug  Administration,  National  Security  Agency,  and  Logistics 
Management  Institu  e-in  addition  to  those  from  the  participating 
activities. 

Attainment  of  the  objectives  of  the  Conference  was  based  on 
the  pre- conference  and  conference  activities  of  nine  panels  or  working 
groups,  which  examined  in  depth,  specific  aspects  of  the  management 
of  the  military  quality  and  reliability  programs.  The  panels  considered 
problems  concerned  with  quality  and  reliability  management  during 
the  development  of  operational  requirements,  materiel  development  and 
production,  and  storage  and  maintenance  during  the  use  phases.  The 
conferees  also  discussed  the  problems  associated  with  the  calibration 
of  equipment  and  the  quality  of  software  to  support  materiel  acquisition 
and  use. 

It  is  anticipated  that  these  Proceedings  will  serve  not  only  as 
a  cross-sectional  record  of  Department  of  Defense  quality  and 
reliability  management  thinking  as  of  August  1966,  but  more 
importantly  as  a  tool  to  help  shape  the  action  of  quality  and  reliability 
improvement  programs  of  the  future.  To  this  end,  166  recommendations 
for  action  resulted  from  the  panel  deliberations.  The  principal 
recommendations  were  presented  orally,  by  the  panel  chairmen,  to 
the  Department  of  Defense  Quality  and  Reliability  Assurance  Council 
and  other  high  ranking  representatives  of  the  Department  of  Defense  on 
4  August  1966. 

The  Proceedings  have  been  prepared  in  two  volumes.  Volume  I 
contains  the  keynote  addresses  and  summaries  of  the  panel  reports; 
Volume  II  contains  full  panel  reports. 


HORACE  R.  LOWERS 
Conference  Director 
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SUMMARY  REPORTS  OF 


CONFERENCE  PANELS 


REPORT  OF  PANEL  1 


TITLE:  Quality  and  Reliability  Management  Concepts 


OBJECTIVE: 


*  c 

t jto  determine  those  objectives,  concepts,  policies 
and  uniform  procedures  essential  to  effective 
quality  and  reliability  management  within  the 
Department  of  Defense .  ;  ; 


/ 


i  \ yfysi; 


TOPICS  DISCUSSED: 


1.  The  Problem  and  Its  Environment 


2.  Policy 

3.  Communication 


4.  Organization 

5.  Resources 

6.  Discipline 

7.  Integrated  Engineering 

8.  Incentive 
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GENERAL  BACKGROUND: 


THE  PROBLEM  AND  ITS  ENVIRONMENT 

The  quality  of  today's  products  in  use  within  the  Department 
of  Defense  is  generally  good.  This  is  borne  out  by  use  throughout 
the  world,  including  Vietnam.  The  occasional  failure  which  does 
occur  is  usually  remedied  without  serious  consequence.  Nonetheless, 
further  development,  integration  and  better  utilization  of  the  quality, 
quality  assurance,  reliability,  reliability  assurance,  maintainability 
( QARAM)  *  disciplines  might  lessen  the  admittedly  high  cost  of 
reasonably  high  quality,  or  provide  improved  quality  at  little 
additional  cost.  There  is  no  adequate  basis  for  quantitative  comparisons. 
Intensive  system  analysis  directed  specifically  to  this  end  is  required 
to  quantitatively  evaluate  these  potentials. 

Technology  is  moving  far  more  rapidly  today  than  ever  before. 

It  has  made  likely  the  situation  that  a  product  can  be  obsolete  before 
it  comes  off  t^e  production  line  if  old  methods  of  in-series  production 
are  used,  i.  e. ,  conception,  research  and  development,  engineering, 
pilot  plant  operation,  prototype,  testing,  production.  Today's 
environment  is  a  telescoping  of  these  life  cycle  phases  into  a  group 
of  semi-parallel  operations  in  which  each  succeeding  ^ifc  cycle  phase 
is  started  well  before  the  preceding  one  is  completed.  Under  such 
circumstances,  it  appears  mandatory  that  there  be  maximum  use  of 
the  QARAM  disciplines  to  insure  that  the  product  does  do  its  job 
despite  the  telescoping  and  other  program  constraints. 

Problem  understanding  has  been  considerably  hampered  by 
poor  definitions  of  the  basic  te  rminology  in  use;  for  example, 
definitions  of  quality,  reliability  and  maintainability.  Many  such 
definitions  currently  exist.  Since  many  definitions  exist,  many 
interpretations  exist.  Further,  the  communication  of  the  QARAM 
ideas  has  been  ineffective  due  to  large  amounts  of  communication 
which  are  duplicative  in  many  areas  but  leave  substantial  gaps  in 
others.  Finally,  there  are  many  factors,  each  having  an  extremely 
broad  spectrum  of  variation  to  be  considered  in  formulating  an 
integrated  approach  to  the  QARAM  problem.  One  such  factor  is 
product  complexity,  in  which  the  spectrum  ranges  from  nuts  and 
bolts,  and  shoelaces,  through  trucks,  tanks,  aircraft,  and  high 
performance  submarines  to  even  more  complex  systems.  Other 
factors  of  concern  are  the  design  life  of  the  product,  the  production 

1  The  use  of  QARAM  in  this  report  is  orkly  lor  the  purpose 
of  expediency. 
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quantity,  and  the  reliability  requirement  (based  on  product  failure 
consequences). 

POLICY 

Despite  a  fairly  large  number  of  OSD  policy  directives  on  the 
street  supplemented  by  subordinate  headquarters  directives,  there 
exist  substantial  gaps  as  well  as  overlaps  in  policy  directives.  It 
appears,  therefore,  that  the  policy  statements  of  the  Office  Secretary 
of  Defense  require  revision  and  condensation. 

The  organizational  and  product  role  of  QARAM  must  be  properly 
defined.  There  must  be  full  identification  and  auditability  of  the  QARAM 
role.  This  QARAM  role  prior  to  production  of  the  product  is  one  of 
"think”  and  establishment  of  realistic  product  attributes;  in  subsequent 
phases  of  the  life  cycle,  it  is  a  "do"  function  concerned  essentially 
with  the  disciplines  of  measurement,  enforcement,  monitoring,  and 
feedback. 

The  governments  role  in  the  spectra  of  product  life,  product 
complexity  and  product  delivery  date  must  be  defined  more  clearly, 
as  must  be  the  vendor's  role.  There  needs  to  be  in  the  policy  statement, 
a  requirement  for  a  disciplined  plan  of  action,  which  includes  integrated 
engineering  as  one  of  its  basic  concepts.  Industrial  funding  operations 
of  the  government  activities  should  be  provided  as  a  basic  costing 
element  in  conjunction  with  the  relaxation  of  conflicting  administrative 
constraints.  Finally,  incentive  motivation  should  be  high  in  the 
requirements  for  each  policy  directive  emanating  from  the  Office 
Secretary  of  Defense. 

COMMUNICATION 

Policy  dissemination  for  problem  resolution  has  not  been 
satisfactory  because  the  definitions  of  quality,  quality  assurance, 
reliability,  reliability  assurance  and  maintainability  have  been 
inconsistent  and  the  proliferation  thereof  has  resulted  in  misinterpre¬ 
tation  and  misunderstanding.  It  would  be  well  to  examine  these  essential 
elements  and  require  that  standard  definitions  be  established  and  used 
throughout  the  Department  of  Defense.  There  is  also  need  to  examine 
the  QARAM  directives,  which  art  currently  outstanding.  We  reviewed 
more  than  150  such  directives  involving  many  thousands  of  pages  of 
directive  material.  When  these  are  added  to  the  various  additional 
requirements  of  specifications,  it  becomes  quite  clear  that  compliance 
is  very  difficult,  if  not  impossible,  for  field  activities  and  contractors. 
Condensing  and  collating  appear  to  be  a  very  profitable  field  for 

improving  communications. 
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There  needs  also  to  be  an  information  gathering  system  which 
could  be  either  centralized  as  in  the  Defense  Documentation  Center 
concept  or  decentralized  in  various  activities  using  a  common  access 
approach  through  common  forms  or  through  interpreting  computer 
connections*  This  information  gathering  operation  should  include 
inforim  tion  on  vendor  capability,  vendor  performance,  pro  duct 
quality,  product  reliability,  product  maintainability  and  on  the  resources 
of  the  various  government  activities,  particularly  in  the  field  of  QARAM. 

ORGANIZATION 

Since  the  problem  is  so  broad  and  the  spectra  so  diverse,  it  is 
unlikely  that  a  single  organizational  structure,  position  or  element  for 
the  QARAM  technique  would  provide  a  satisfactory  solution.  What  is 
required,  however,  is  full  QARAM  identification  and  auditability  in 
each  organization  and  in  each  product  life  cycle  to  insure  that  this 
important  element  is  properly  provided.  QARAM  should  be  placed 
organizationally  to  assure  authority  and  management  access  commensurate 
with  other  functional  disciplines.  There  is  a  need  to  integrate  the 
splinter  disciplines,  which  have  tended  to  become  overemphasized 
and  to  distort  current  products.  These  disciplines,  including 
standardization  engineering,  human  engineering,  safety  engineering, 
etc. ,  should  be  restored  to  the  basic  integrated  engineering  structure, 
both  in  the  Department  of  Defense  and  the  various  vendors'  operations 
to  insure  maximum  effectiveness. 

Within  the  Office  Secretary  of  Defense,  the  organizations  of  the 
Defense  Director  for  Research  and  Engineering,  and  Assistant  Secretary 
for  Installations  and  Logistics  appear  adequate  to  handle  the  QARAM 
philosophy.  It  is  understood  that  the  basic  QARAM  "think"  capability, 
that  is,  determination  of  quality  of  the  product  through  product  attributes 
(considering  quality,  reliability  and  maintainability  as  product 
attributes)  is  a  function  of  engineering  and  that  the  basic  QARAM  "do" 
function,  which  is  enforcement,  is  a  basic  Installations  and  Logistics 
responsibility;  (that  is  the  primary  monitoring  and  enforcement  of  the 
attributes  called  out  by  engineering). 

In  all  organizational  matters,  the  audit  function  must  be  fully 
represented,  fully  available  and  used.  Every  organization  involved 
requires  such  capability.  It  should  be  provided. 

RESOURCES 

Within  the  next  five  years  there  will  be  a  serious  reduction  in 
this  capability  due  to  the  age  of  many  of  the  personnel  involved  in  the 
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QARAM  operations  in  the  field.  This  falling  off  of  World  War  II  and 
Korean  War  personnel  requires  replacement  by  other  personnel  resources. 
There  is  a  need  for  an  acquisition  plan  which  includes  the  necessary 
training  and  cross -fertilization  of  personnel  so  that  they  can  be 
completely  effective.  Further,  there  is  a  need  for  a  retention  of 
personnel  plan  which  would  provide  for  the  requisite  career  patterns 
to  insure  that  personnel  currently  on  board  or  acquired  remain  with  the 
government.  (This  is  to  be  covered  by  Panel  7.)  Finally,  there  is  a 
need  to  consider  the  manner  in  which  the  personnel  resources  are 
utilized  under  current  types  of  products.  There  is  duplication  of 
effort  among  government  activities  and  among  the  various  vendors 
and  sub-vendors.  Such  duplication  of  effort  is  undesirable,  is 
unnecessarily  costly,  time-consuming  and  results  in  confusion. 

There  is  a  need  to  develop  plans  to  prevent  such  undesirable  duplication. 

DISCIPLINE 


The  success  of  the  Office  Secretary  of  Defense  plan  for  QARAM 
resides  not  only  in  having  a  good  plan,  but  also  in  those  additional 
elements  which  we  collectively  call  the  "discipline'1  discipline.  The 
fifth  definition  in  the  American  Collegiate  Dictionary  is  "A  state  of 
order  maintained  by  training  and  control. "  We  are  considering  such  a 
situation  in  this  concept.  The  maintenance  of  a  state  of  order  indicates 
that  there  was  a  concept,  a  development  of  a  plan,  a  mode  for 
implementation,  a  promulgation,  an  execution,  and  auditing  and  finally 
an  improvement  cf  the  plan,  and  that  these  acts  were  performed  by 
personnel  of  requisite  skills,  background,  training  and  desire.  Thus, 
the  "discipline"  discipline  involves  the  total  operation  of  thinking  and 
doing. 


While  QARAM  has  no  specific  plan  because  of  the  immense 
complexity  of  the  Department  of  Defense  spectra  of  products,  life 
cycle  products  and  product  age,  personnel  available,  facilities  and 
equipments  available,  it  is  necessary  that  the  concept  of  discipline  be 
re-emphasized  in  operations.  The  problem  is  sufficiently  important 
for  a  basic  conceptual  plan  to  be  developed,  promulgated,  executed  and 
audited  for  the  execution  of  the  Office  Secretary  of  Defense  concepts 
for  product  management. 
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INTEGRATED  ENGINEERING 


The  "think"  phase  of  product  development  is  all  too  often 
considered  to  include  only  the  conception  and  design  of  the  product. 
However,  the  "think"  phase  for  full  product  exploitation  must  cover 
the  entire  life  cycle.  "Think"  and  engineering  are  reasonably 
synonymous  in  product  development.  Engineering  is  an  inclusive 
term  which,  in  essence,  provides  for  professional  personnel  to  apply 
their  talents  in  a  disciplined  way  to  a  specific  aspect  of  a  product's 
attributes  or  life.  Basic  engineering  should  think  out  the  entire  life 
cycle  of  the  product.  Because  of  the  nature  of  the  OSD,  there  is  no 
requirement  for  large  numbers  of  engineers  operating  in  a  basic 
engineering  environment  to  develop  a  product  or  a  series  of  products. 

Due  to  the  use  of  various  secondary  disciplines  or  management 
disciplines,  there  has  been  acquired  vithin  the  Office  Secretary  of 
Defense  a  number  of  splintered  discipline  capabilities,  for  example, 
safety  engineering,  value  engineering,  standardization  engineering, 
standards  engineering,  etc.  The  proliferation  of  these  splintered 
engineering  disciplines  has  unfortunately  sometimes  resulted  in  undue 
emphasis  being  placed  on  certain  aspects  of  product  capabilities  or 
product  attributes.  The  emphasis  results  in  distortion  of  the  product 
in  many  cases,  improper  application  of  rerources  to  the  product. 

There  is  a  great  need  to  restore  these  splintered  engineering  disciplines 
to  a  basic  integrated  engineering  framework.  While  it  is  not  possible 
within  the  Office  Secretary  of  Defense,  such  is  possible  within  most 
of  its  field  units.  Unfortunately,  by  directives  from  the  Office  Secretary 
of  Defense,  splintered  engineering  disciplines  in  field  activities 
usually  are  organized  outside  the  basic  engineering  operations.  As 
such,  they  cannot  contribute  effectively,  and  often  tend  to  distort 
rather  than  to  make  effective.  The  "think"  portion  of  the  QARAM 
discipline,  among  others,  needs  to  be  included  in  the  basic  engineering 
effort  for  optimizing  the  products  attributes  for  the  product's  entire 
life  cycle.  These  attributes  do,  of  course,  include  quality,  reliability 
and  maintainability. 

INCENTIVE 


In  a  situation  as  complex  as  that  in  which  the  Office  Secretary 
of  Defense  operates,  there  can  be  no  single  management  discipline,  no 
single  engineering  discipline,  and  really  no  combination  of  these  which 
provides  basic  policy  which  would  be  successful  in  all  situations.  The 
Office  Secretary  of  Defense  and  other  controllers  of  large  complex 
efforts  must  go  to  the  basic  motivational  disciplines  which  affect  men. 
One  such  is  incentive;  incentive  brings  out  the  best  in  man.  It  is  a 
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continuing  type  of  motivation.  Department  of  Defense  directives,  policies 
and  efforts  to  acquire  maximum  effectiveness  ir>  the  QARAM  field  (as 
well  as  in  all  others)  should  be  by  means  of  maximum  use  of  incentive., 
This  is  desirable  not  only  for  the  vendors  of  products  desired  by  the 
Department  of  Defense  but  also  for  its  own  personnel  in  field  activities, 
headquarters,  and  in  the  Office  Secretary  of  Defense  itself.  A  full 
range  of  incentive  motivational  devices  should  be  explored,  studied, 
developed  and  implemented.  The  use  of  an  incentive  approach  could 
provide  more  benefit  than  all  of  the  preceding  efforts  to  describe  and 
acquire  a  better  system, 

SUMMARY  OF  RECOMMENDATIONS: 


Panel  1  recommends  that: 

1.  The  Office  Secretary  of  Defense  establish  and  chair  a 
committee  composed  of  representatives  from  the  Department  of  Defense 
complex  to  set  forth  the  format  for  a  handbook  utilizing  the  Quality 
Assurance,  Reliability  Assurance  and  Maintainability  (QARAM)  discipline. 
The  handbook  should  be  prepared  by  a  qualified  management  source  under 
the  general  guidance  of  this  committee  with  a  target  date  of  1  July  1967 

for  completion.  It  should  include  policy,  general  procedures,  management 
concepts  and  techniques,  and  an  appendix  listing  of  all  directives  and 
implementing  instructions  in  the  QARAM  field. 

2.  Department  of  Defense  Directive  4155.11  be  revised  to  cover 
the  QARAM  discipline  with  emphasis  cn  identification  and  auditability 
of  the  product  and  organizational  resources.  Revision  should  include 
incentive,  brevity,  communication,  contractor  performance  evaluation 
(CPE),  unsatisfactory  material  reporting  (UMR),  and  language.  This 
effort  should  include  the  review  of  all  listed  Department  of  Defense 
directives  for  conformance  with  QARAM  or  elimination. 

3.  A  directive  be  issued  to  support  the  QARAM  discipline  in 
the  area  of  Office  Secretary  of  Defense  (DDR&tE)  jointly  with  Office 
Secretary  of  Defense  (I&L)  reaffirming  that  the  determination  of 
product  is  the  function  of  DDR&cE  and  the  fulfillment  of  the  QARAM 
requirements  if  the  function  of  I&L.  Further,  that  the  current  splintered 
engineering  disciplines  which  are  separately  organized  and  supported  be 
reintegrated  into  a  basic  engineering  structure  to  support  the  life  cycle 
of  the  product. 

4.  The  concept  of  industrial  funding  be  utilized  to  provide 
incentive  auditability  and  costing.  It  is  further  recommended  that  a 
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study  be  initiated  to  determine  the  feasibility  of  providing  the  field 
managers  and  Defense  Contract  Administration  Services  with  a 
product- oriented  funding  structure.  The  panel  determined  that  this 
type  of  funding  approach  provides  the  best  incentive  to  efficient  and 
economical  operation.  It  also  allows  for  better  management. 

5.  Office  Secretary  of  Defense  study  the  matter  of  discipline 
in  the  QARAM  field  and  develop  a  plan  for  implementing  its  needs  in 
this  area  by  promulgating!  executing!  auditing!  and  improving  the  plan. 

6.  Office  Secretary  of  Defense  study  the  problem  of  simultaneous 
audits  of  vendors  for  the  same  purpose  between  Department  of  Defense 
and  other  government  agencies. 

7.  Department  of  Defense  make  a  study  of  the  broad  application 
of  the  incentive  approach  as  a  means  of  improving  management 
effectiveness. 


NOTE:  The  above  seven  recommendations  represent  a  summarisation 
of  Panel  l's  55  recommendations  contained  in  Volume  II. 
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report  of  PANEL  2 


TITLE:  Quality  and  Reliability  Management  in  the  Development  Phase 
OBJECTIVE: 

To  improve  the  methods  within  the  Department  of  Defense 

of  establishing  requirements  for  and  obtaining  materiel 

with  the  required  quality  and  reliability  during  the 

development  program. 

TOPICS  DISCUSSED^ 

1.  Adequacy  of  Operational  Requirements 

2.  Stratification  of  Inherent/Operational  Characteristics 

3.  Reliability  Demonstration  Requirements  during 
Contract  Definition 

4.  Reliability/ Quality/ Cost/Time/ Performance  Trade-offs 

5.  Storage  Degradation  Factors  as  Design  Parameters 

6.  Incentives  to  Minimize  Adverse  Reliability  and 
Quality  Trade-offs 

7.  Parts  for  High  Reliability/ Long  Life  Applications 

8.  Time  Phasing  of  Reliability  and  Quality  Assurance 
Functions  in  the  Program  Cycle 

9.  Specification  and  Specification  Guidelines 

10.  Resources  for  Advancing  Reliability  and  Quality 
Technology,  Reliability  and  Quality  Programs, 
and  Reliability  Demonstration 


SUMMARY  OF  RECOMMENDATIONS: 


Panel  2  recommends  that: 

1.  Operational  commanders  be  encouraged  to  include  operational 
information  that  affects  reliability  and  maintainability  design. 

2.  During  the  conceptual  phase,  reliability  and  maintainability 
requirements  be  refined  by  analysis  and  study  of  alternatives.  After 
the  SOR  or  OMR  is  promulgated,  trade-offs  should  not  be  made  without 
a  complete  analysis  and  synthesis  comparing  all  the  alternatives  in  the 
original  proposed  technical  approach  and  others  that  may  have  become 
feasible  in  the  interim. 

3.  An  examination  be  made  of  existing  data  reporting  systems 
to  determine  their  capability  to  stratify  the  inherent  and  operational 
characteristics. 

4.  ,,K"  factors  associated  with  the  operational  use  of  selected 
combat  systems  and  equipments  be  defined. 

5.  The  services  establish  or  place  more  emphasis  on  assurance 
programs  that  will  minimize  the  degradation  of  inherent  reliability  of 
the  design  during  the  production,  operational  deployment  and  logistics 
support  phases. 

6.  The  requirements  of  MIL-STD-785  be  adequately  detailed 
during  contract  definition  and  in  all  succeeding  program  phases. 

7.  Continued  research  be  sponsored  by  the  services  in 
improving  cost-effectiveness  techniques  and  modeling  which  consider 
reliability  and  quality  as  elements  of  effectiveness. 

8.  All  services  continue  their  efforts  to  refine  pertinent  data 
and  techniques  for  use  in  cost-effectiveness  trade-offs. 

9.  Management  ensure  utilisation  of  reliability -quality  cost- 
effectiveness  trade-offs  in  the  decision  making  process. 

10.  The  services  conduct  studies  on  systems  and  equipments  to 
determine  the  causes  and  extent  of  reliability  degradation  in  storage 
so  that  program*  can  be  developed  to  minimise  degradation. 
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11.  Research  be  supported  to  investigate  accelerated  testing 
techniques  for  predicting  and  measuring  reliability  degradation  in 
storage. 


12.  Incentives  be  designed  to  minimize  adverse  trade-offs  which 
affect  quality  and  reliability. 

13.  Reliability  and  qiality  incentives  be  structured  into  contracts 
where  possible. 

14.  Development  contracts  specify,  to  the  maximum  extent 
possible,  the  use  of  available  ER  grades  of  parts  in  all  critical  sub¬ 
systems  consistent  with  their  requirements. 

15.  One  hundred  percent  acceptance  screening  be  required  for 
all  electronic  parts  not  covered  by  ER  specifications  and  used  in 
critical  subsystems.  Provision  should  be  made  to  accumulate  and 
disseminate  data  generated  from  these  tests. 

o 

16.  The  ER  parts  specification  programs  be  expanded  to  other 
electrical  and  electronic  parts,  including  microcircuits.  Also,  high 
usage  mechanical  parts  should  be  studied  to  determine  the  feasibility 
of  ER  specifications. 

17.  Department  of  Defense  investigate  the  methods  of  qualifying 
microcircuits  based  on  commonality  between  different  types  of  circuits. 

18.  Department  of  Defense  determine  the  feasibility  of  screening 
procedures  for  mechanical  parts,  similar  to  those  for  electronic  parts. 

19.  Department  of  Defense  determine  the  feasibility  of  expanding 
the  use  of  ER  parts  in  other  than  critical  subsystems  to  attain  economic 
production  runs  and  to  ensure  availability. 

20.  Preparation  of  the  proposed  Department  of  Defense  /National 
Aeronautics  and  Space  Administration  Handbook  of  parts  screening  and 
burn-in  procedures  (for  electronic  parts)  be  accelerated. 

21.  The  services  develop  appropriate  guidelines  which  permit 
the  operating  agencies  to  develop  critical  reliability  and  quality 
assurance  activities  and  requirements  tailored  to  the  specific  projects 
and  program  phases. 
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22.  The  services  place  more  emphasis  on  making  reliability 
and  quality  assurance  functions  complementary  to  ensure  completeness, 
timeliness  and  cost-effectiveness. 

23.  The  services  explore  the  feasibility  of  expanding  the  use  of 
production  testing  techniques  for  specifications. 

24.  The  Office  of  the  Secretary  of  Defense  provide  the  necessary 
resources  in  manpower  and  funding  to  expand  efforts  for  the  advancement 
of  reliability  and  quality  technologies,  and  ensure  that  each  project 
identifies  adequate  resources  for  the  R&Q  program  and  its  proper 
demons  tration. 
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REPORT  OF  PANEL  3 


TITLE:  Quality  and  Reliability  Assurance  in  the  Production  Phase 

OBJECTIVE: 

To  improve  the  capability  within  the  Department  of 

Defense  to  establish  requirements  for  and  to  obtain 

materiel  possessing  the  necessary  quality  and 

reliability  during  the  production  phase . 

TOPICS  DISCUSSED: 

1.  Adequacy  and  Completeness  of  Technical  Documentation 
for  Procurement 

2.  Use  of  Systems  Specifications  MIL-Q-9858A,  MIL-I-45208, 
MIL- C- 45662 

3.  Cost  Considerations  in  Production  Quality  Assurance 

4.  DOD  Quality  and  Reliability  Assurance  Technical 
Procedures 

5.  Utilization  of  Customer  Deficiency  Reports 

6.  Quality  Assurance  Role  in  Overall  Contractor  Performance 
Evaluation 

7.  Role  of  Incentives  and  Warranties  in  Improving  Quality 

8.  Role  of  Government  Quality  Assurance  in  Subcontracts 

9.  Contractor  Quality  Control  versus  Government 
Quality  Assurance 

10.  Impact  of  Procuring  Activity  Product  Evaluation  on 
Administering  Agencies 

11.  Adequacy  of  Government  Control  over  Nonconforming 
Supplies 
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12.  Quality  Assurance  Response  to  Fluctuating  Workloads 

13.  Quality  Requirements  for  Small  Dollar  Procurements 

14.  Procuremext  Quality  Assurance  Interfaces 


SUMMARY  OF  RECOMMENDATIONS: 


Panel  3  recommends  that: 

1.  Office  Secretary  of  Defense  policies  be  issued  to  require 
that  quality  assurance  personnel  (a)  review  technical  documentation 
for  adequacy  prior  to  procurement,  (Id)  provide  the  required 
quality  assurance  contract  clause  input,  and  (6)  arrange  for  the 
conduct  of  post-award  conferences  on  major  procurements  to  assure 
an  understanding  of  product  quality  requirements. 

2.  Department  of  Defense  activities  continue  to  stress  the 
necessity  for  sound  configuration  management  practices,  including 
adequate  testing  and  evaluation  of  both  materiel  and  technical 

do  cum  entation. 

3.  Department  of  Defense  activities  assure  that  quality 
assurance  personnel  participate  in  the  development  of  technical 
documentation. 

4.  A  Department  of  Defense  committee  be  established  to 
determine  what  additional  quality  assurance  contract  clauses  would 

be  beneficial  for  inclusion  in  ASPR,  to  improve  technical  documentation. 

5.  Office  Secretary  of  Defense  develop  and  publish  uniform 
and  definitive  guidance  for  evaluating  the  effectiveness  of  a  supplier's 
implemented  program  simultaneously  with  issuance  of  the  program 
specification. 

6.  Office  Assistant  Secretary  of  Defense  (Installations  and 
Logistics)  sponsor  a  study  group  to  explore  all  facets  of  production 
quality  costs  of  contractor  and  government,  and  to  identify  those 
elements  of  cost  that  should  be  maintained.  This  group  should  also 
develop  a  uniform  policy  relative  to  charging  contractors  for 
government  costs  associated  with  reinspections  of  rejected  materiel 
and  for  the  application  of  cost  considerations  in  connection  with  the 
acceptance  of  nonconforming  materiel. 

7.  Office  Secretary  of  Defense  establish  an  effective  mechanism 
to  assure  that  a  single  series  of  quality  and  reliability  assurance  policies 
and  procedures  is  officially  published  and  kept  up  to  date. 
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8.  Office  Secretary  of  Defense  establish  a  project  for  the 
development  of  a  standard  DD  Form  and  appropriate  procedures 
to  be  used  as  a  transmittal  sheet  for  submitting  deficiency  reports 
to  contract  adminstration  services  components. 

9.  All  elements  of  Department  of  Defense  review  their 
deficiency  reporting  systems  to  assure  that  they  provide  for  an 
expeditious  flow  of  deficiency  reports  through  quality  assurance 
channels  to  either  design  agencies  (design  deficiencies)  or  contract 
administration  activities  (quality  deficiencies)  as  appropriate. 

10.  Office  Secretary  of  Defense  establish  a  task  group  composed 
of  quality  assurance  personnel  to  evaluate  the  contract  administration 
quality  assurance  programs  now  carried  out  by  the  several  Department 

of  Defense  elements,  and  develop  a  Department  of  Defense -wide  program 
at  the  earliest  practicable  date.  Procedures  and  resultant  data  recording 
and  reporting  methods  developed  should  be  compatible  with  Contractor 
Performance  Evaluation  (CPE)  program  needs. 

11.  Office  Secretary  of  Defense  establish  a  task  group  of  quality 
assurance  personnel  to  develop  specific  quality  assurance  factors  for 
inclusion  in  the  Department  of  Defense  CPE  program. 

12.  Office  Secretary  of  Defense  establish  unifrom  quality 
considerations  for  the  Department  of  Defense-wide  CPE  program 
which  include  the  best  features  of  departmental  CPE  programs. 

13.  Office  Secretary  of  Defense  request  the  ser  /ices  and 
Defense  Supply  Agency  to  furnish  the  results  of  their  experience 
with  incentives  and  warranties  relative  to  quality  and  reliability, 
in  crder  to  (1)  identify  quality  and  reliability  evaluation  elements 
during  production  for  performance  incentive  contracts,  (2)  assess 
the  overall  impact  of  warranties  with  emphasis  in  the  areas  of  cost, 
relationship  to  all  types  of  incentives  and  appropriate  application, 
and  (3)  use  the  data  for  developing  guidelines  for  use  of  procuring 
activities. 

14.  Office  Secretary  of  Defense  should  incorporate  the  following 
concepts  in  policy  and  procedural  directives  in  order  to  clearly  define 
the  government  quality  assurance  role  in  subcontracting. 

a.  Hold  prime  contractors  completely  responsible  for  the 
quality  of  supplies  received  from  their  subcontractors 
and  vendors  through  definitive  contractual  language, 
nothwithstanding  government  source  inspection. 
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b.  Require  government  quality  assurance  review  of  prime 

contractor  proposals  for  subcontracting. 

c.  Strengthen  government  review  of  prime  contractor 
quality  systems,  including  written  procedures  for  control 
over  the  quality  of  subcontracted  supplies. 

d.  Define  the  specific  circumstances  under  which  government 
source  inspection  may  be  applied. 

e.  Provide  that  requests  for  government  source  inspection 
contain  complete  identification  of  specific  products, 
processes,  critical  characteristics  and  other  requirements 
to  be  examined. 

15.  The  ASPR  Committee  assure  the  development  of  standardized 
procurement  quality  assurance  procedures  reflecting  the  basic  policies 
of  Department  of  Defense  concerning  contractor  responsibility  for 
control  of  product  quality.  Procedures  should  be  adaptable  to  different 
products  and  allow  for  government  effort  commensurate  with  demonstrated 
contractor  effectiveness. 

16.  No  action  be  taken  by  Office  Secretary  of  Defense  at  this 
time  to  standardize  the  procuring  activities’  approach  to  product 
evaluation. 


17.  Office  Secretary  of  Defense  develop  and  publish  uniform 
policy  and  procedural  guidance  for  contractor  and  government  control 
over  nonconforming  supplies  and  materials. 

18.  Office  Secretary  cf  Defense  change  the  present  policies  and 
procedures  concerned  with  procurement  quality  assurance  manpower 
to  provide  a  capability  to  respond  to  fluctuating  workloads. 

19.  Office  Secretary  of  Defense  issue  a  policy  statement 
establishing  the  conditions  for  the  services  and  Defense  Supply  Agency 
to  specify  directed  inspection  activity  at  source.  This  policy  should 
give  cognizance  to  the  relative  importance  of  materials  procured  by 
the  services  and  Defense  Supply  Agency  in  comparison  to  the  total 
military  mission  of  Department  of  Defense,  and  encourage  efforts  to 
minimize  undirected  mandatory  inspections. 

20.  Office  Secretary  of  Defense  explore  the  practicality  of 
funding  the  direct  labor  involved  in  the  Contract  Administration  Services 
field  quality  assurance  effor‘  as  an  administrative  expense  of  procurements. 
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21.  An  ASPR  case  be  established  to  investigate  simplified 
purchase  procedures  with  the  objective  of  allowing  maximum  use  while 
maintaining  necessary  government  quality  protection. 

22.  The  Deputy  Secretary  of  Defense  establish  a  task  group 
to  develop  a  total  quality  and  reliability  program  covering  the  entire 
product  life  cycle,  or  to  develop  a  system  that  will  assure  that 
adequate  consideration  is  given  to  quality  and  reliability  requirements 
throughout  the  product  life  cycle. 

23.  Department  of  Defense  recognize  the  two  methods  of  developing 
weapon  systems  (in-house  and  industry);  and  that  there  is  a  distinct 
difference  in  the  technical  documentation  that  results  from  each. 

24.  Assistant  Secretary  Df  Defense  (Installations  and  Logistics) 
assure  that  timely  action  is  taken  to  develop  a  uniform  set  of  quality 
assurance  procedures  for  application  to  the  bulk  of  Department  of 
Defense  procurements  in  consonance  with  the  intent  of  ASPR  1-108. 


REPORT  OF  PANEL  4 


TITLE:  Quality  Assurance  in  Storage  Operations 

OBJECTIVE: 

To  assure  the  quality  and  reliability  of  materiel  being 
issued  for  use  from  storage  operations. 


TOPICS  DISCUSSED: 

1.  Quality  of  Material  Upon  Receipt  by  Storage  Activities 

2.  Quality  Deterioration  of  Material  in  Storage 

SUMMARY  OF  RECOMMENDATIONS: 


Panel  4  recommends  that: 

1.  The  concept  of  "storageability"  be  defined  and  standardized 
in  the  Research  and  Development  phase  and  that  these  requirements  be 
given  visibility  and  incorporation  within  the  production  technical  data 
package. 

2.  Department  of  Defense  take  necessary  action  to  (a)  eliminate 
or  reduce  drastically  the  awarding  of  "acceptance  at  destination" 
contracts,  including  as  a  minimum  all  items,  which  if  defective,  would 
cause  malfunctions  or  losses  of  military  weapons  and  combat  equipments, 
or  (b)  amend  Department  of  Defense  Directive  4155.11  to  require  all 
storage  activities  to  inspect  and/or  test  material  received  under 
"acceptance  at  destination"  contracts  at  the  time  of  receipt  from 
vendors/contractors,  and  provide  the  resources  to  do  so, 

3.  Department  of  Defense  Directive  4155.11  (Paragraph  V.  B) 
be  amended  to  delete  authorization  to  combine  quality  assurance 
functions  with  other  storage  operations,  if  the  intent  of  the  directive 
is  to  require  independent  quality  and  reliability  management. 
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4.  Department  of  Defense  Directive  4155.11  be  amended  to 
specify  those  quality  control  and  reliability  functions  to  be  performed 
by  a  storage  quality  control  organization. 

5.  Material  being  procured  for  storage  activities  (or  for  the 
operating  forces)  be  marked  for  identification  by  Federal  Stock  Number, 
where  such  numbers  have  been  assigned. 

6.  Material  managers  such  as  Commodity  Managers,  Project 
Managers,  Project  Officers,  System  Program  Director,  etc. ,  be 
required  to  provide  storage  activities  with  Storage  Serviceability 
Standards  for  material  receipt,  storage  and  issue  in  storage  operations. 
These  standards  must  define  the  minimum  level  to  which  deterioration 
can  progress  without  impairment  of  serviceability  and  user  satisfaction. 

7.  Utilization  of  storage  and  user  test  and  evaluation  data  be 
made  mandatory  and  that  necessary  arrangements  be  made  at  the 
Department  of  Defense  level  to  utilize  skills  and  facilities  unique  to 
any  one  service  by  all  services  in  the  quality  evaluation  of  stored  and 
service-held  material. 

8.  Results  of  routine  periodic  storage  testing  of  complex 
items  (particularly  electronic)  be  investigated  on  a  selective  basis  and 
that  the  reliability  degradation  from  these  tests  be  weighed  versus 
the  quality  assurance  gains. 
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REPORT  OF  PANEL  5 


TITLE:  Quality  Assurance  in  Maintenance  Operations 

OBJECTIVE: 

To  assure  that  material  reconditioned,  maintained 
and  modified  for  the  Military  Services  meet  requisite 
quality . 


TOPICS  DISCUSSED: 

1.  Consistent  Quality  Assurance  at  all  Echelons  of 
Maintenance 

2.  Continuity  of  Consecutive  Modifications 

3.  Inspection  Cognizance  for  Contract  Maintenance 

4.  Reliability  Management  relating  to  Maintenance 

5.  Relationship  between  Reliability  Analysis  and 
Preventive  Maintaine nance 

6.  Reconditioning  Standards 

7.  Quality  Standards  for  MAS  (Military  Assistance  Sales) 
and  Grant  Aid  Materiel 

8.  Improved  Reconditioning/Serviceability  Standards  for 
Attribute  Characteristics 

9.  Use  of  Contractor  Rejected  Material  in  Maintenance 
Activities 

10.  Impact  of  Contract  Waivers  on  Maintenance 

11.  Quality  Control/Inspection  Skills  Gap 
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12.  Reducing  Maintenance  Workload  Through  More  Complete 
•'Use"  Instruction 

13.  Maintenance  Technical  Data 

14.  Deficiency  Data  Feedback 


GENERAL  BACKGROUND: 


The  areas  identified  for  consideration  by  this  panel  cover 
organizational  policy  and  procedural  responsibilities  for  the 
maintenance  of  material  after  it  initially  enters  the  Department  of 
Defense  inventory.  Included  are  functions  performed  by  government 
depots,  government  activities  in  support  of  maintenance  and  rebuild 
under  contract,  and  those  maintenance  and  modification  functions 
carried  out  by  or  in  support  of  Military  users. 

During  the  past  two  decades,  many  significant  changes  have 
taken  place  in  the  operation  of  the  maintenance/modification  programs 
throughout  the  Department  of  Defense.  Changes  in  logistic  concepts, 
procurement  and  storage  practices  in  support  of  maintenance  operations, 
and  the  tremendously  increased  costs  and  rapidly  increasing  complexity 
of  defense  equipment  have  had  a  very  profound  effect  on  defense 
maintenance  operations.  These  changes  have  had  considrable  impact 
on  quality  control  in  the  maintenance  area,  and  strengthen  the  need 
for  more  effective  quality  control  operation  in  this  most  vital  area. 

The  facilities,  skills,  and  equipment  required  to  support  material 
maintenance  varies,  depending  upon  the  type  of  equipment  involved, 
and  the  amount  and  complexity  of  the  maintenance/modification  involved. 
The  work  is  categorized  and  scheduled  accordingly,  ranging  from  formal, 
complex  specialized  repair  activities  to  those  routine  actions  performed 
by  the  using  tactical  or  defense  element.  For  purpose  of  this  study, 
all  maintenance  is  identified  as: 

Depot  maintenance 

Contract  maintenance 

Unit  or  organizational  maintenance 

Contract  maintenance  describe,  the  process  whereby  resources 
required  for  the  performance  of  maintenance,  repair,  overhaul  and 
modification  of  material  in  the  Department  of  Defense  inventory,  and 
not  currently  available  within  the  government's  industrial  complex,  is 
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acquired  under  contract  with  commercial  repair  institutions  outside 
the  Government.  In  effect,  contract  maintenance  is  an  extension  of 
the  Government's  organic  maintenance  capability,  and  as  such,  cannot 
be  conveniently  divorced  from  the  administration  of  the  overall  (total) 
program. 

In  past  years,  an  increasing  proportion  of  the  depot  workload 
has  been  accomplished  by  contract.  Defense -wide  figures  are  not 
readily  available,  but  in  the  USAF,  approximately  48-52%  of  the 
maintenance  dollars  is  obligated  to  support  of  contractor  performed 
maintenance  /  modification. 

The  magnitude  of  the  DOD  maintenance  operations  and  the 
importance  of  quality  control  in  the  maintenance  area  is  best  high¬ 
lighted  by  examining  some  of  the  current  statistics  relating  to  mainte¬ 
nance  within  DOD. 

There  is  an  estimated  total  of  4.  5  million  Line  Items  of  equip¬ 
ment  currently  required  to  support  the  US  defense  program.  About  15% 
of  these  are  recoverable  (repairable) .  Of  this  amount,  approximately 
half  can  be  repaired  locally.  The  balance  (7-8%  of  total)  require  depot 
(or  contract)  level  repair  and  must  be  included  in  the  maintenance 
management  workload.  To  provide  the  required  maintenance  support 
of  the  DOD's  estimated  $105  billion  inventory  in  weapons  and  equipment, 
an  annual  budget  approximating  $12  billion  is  required  (based  on  FY  66). 
This  figure  includes  direct  labor  and  replenishment  items  only,  but 
does  not  include  the  cost  of  facilities,  services,  training,  etc.  Con¬ 
sidering  only  labor  and  material,  maintenance  costs  represent  approxi¬ 
mately  25%  of  the  annual  defense  budget,  and  involves  approximately 
900,000  personnel,  or  1/4  of  total  DOD  manpower. 

While  aggregate  manpower  figures  are  not  readily  available,  it 
can  be  shown  that  a  minimum  of  10%,  and  in  some  cases  upwards  of 
16%  of  the  maintenance  workload  is  allocated  to  accomplishing 
inspection/quality  control.  Based  on  a  conservative,  generally 
accepted  ratio  of  10%,  a  minimum  of  $1.  2  billion  and  90,  000  manyears 
(2.4%  of  the  annual  budget  and  2.  5%  cf  the  manpower  resources) 
available  to  the  Dept  of  Defense  are  therefore  required  annually  to 
support  this  maintenance  quality  control  workload.  Because  of  this 
very  heavy  proportionate  commitment  of  DOD  manpower  and  budget, 
it  must  be  concluded  th^l  effective  quality  control  management  is 
essential  for  maximum  operating  cost  effectiveness. 
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SUMMARY  OF  RECOMMENDATIONS: 


Panel  5  recommends  that: 

1.  A  DOD  directive  be  prepared  that  firmly  establishes 
organizational  stature  for  all  echelons  of  maintenance  quality  assur¬ 
ance  compatible  with  generally  accepted  and  proven  methods  of  quality 
assurance  management.  It  should  be  patterned  after,  for  example, 
NavShips  Instruction  5450. 148,  Standard  Naval  Shipyard  Regulations, 
which  recognizes  the  importance  of  organizational  autonomy  for  the 
quality  assurance  function  in  support  of  progressive,  dynamic 
quality  improvement  programs.  In  supplementing  this  recommenda¬ 
tion,  an  objective  operating  manual  or  handbook  should  be  issued 
describing  operations,  responsibilities,  means  of  communication, 
appraisal,  etc.  Finally,  the  problem  of  staffing  this  function  with 
superior  civilian  and  military  personnel  should  be  met  head-on  with 
the  creation  of  formidable  positions  of  executive  importance. 

2.  A  centralized  control  be  established  fcr  all  actions 
immediately  subsequent  to  formal  issuance  of  maintenance  modifications. 
This  control  should  be  in  the  form  of  a  system  that  prevents  modifi¬ 
cations  to  be  “ffected  out  of  sequence  and  must  consider  the  attendant 
problems  contributing  to  the  dilemma  such  as  state  of  completion  of 
fabrication  or  test,  existing  inventory,  recall  of  documents  made 
obsolete  as  the  result  of  the  modification. 

3.  The  Service s/DCAS/CSD  follow  through  on  the  original 
concept  to  reassign  maintenance  contract  administration  to  the  parent 
Service  for  those  contractor  facilities  primarily  engaged  in  specialized 
maintenance,  repair,  overhaul  and  modification. 

4.  Quality  and  reliability  requirements  for  depot  maintained 
items  be  developed  that  relate  in  a  logical  manner  to  those  invoked  at 
the  initial  productive  phases  and  with  due  consideration  given  to  the 
influences  of  usage,  changes  in  mission  objectives,  substitution  of 
different  material,  etc. 

5.  OSD  issue  a  policy  instruction  that  makes  specific  refer¬ 
ence  to  the  working  relationship  between  quality  assurance  and 
reliability  and  maintainability  and  it  should  clarify  and  consolidate 
requirements  now  appearing  in  DOD  Instructions  3200.6,  3200.9, 

4100.  35  and  3231.  1  to  include  all  reparable  systems,  components, 
equipments  procured  by  the  DOD.  This  policy  should  emphasize  the 
importance  of  more  active  participation  of  capable  quality  control 
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personnel  in  the  normal  process  of  work  planning  from  design  through 
ultimate  fabrication  and  test. 

6.  A  statement  of  consistent  policy  be  effected  by  OSD  re¬ 
garding  reconditioning  standards  along  with  adequate  procedural 
quality  assurance  guidance  for  commodity  managers  in  facilitating 
ultimate  development.  Consideration  of  acceptable  wear  tolerances 
for  important  functional  characteristics  should  be  given  in  the 
generation  of  a  standard  format  suitable  for  use  by  both  services  and 
industry. 

7.  Specific  regulations  governing  quality  requirements  be 
provided  for  military  assistance  sales  and  grant  aid  materiel  along 
with  service  directives  that  reflect  these  requirements  as  assurance 
of  mutual  understanding  and  agreement  between  customer  and  vendor. 

8.  Policy  require  the  development  of  interpretive  standards 
in  the  inspection  of  such  attributes  as  color,  finish,  damage,  etc.  A 
clearing  house  should  be  designated  to  function  as  coordinator  for 
development  and  implementation  of  numerous  service-wide  standards. 

A  formal  determination  is  needed  of  the  extent  of  existing  standards, 
both  DOD  and  industry,  along  with  a  formal  refinement  of  present 
practices.  The  Services  and  DSA  should  vigorously  effect  programs 
aimed  at  quantifying  attributes  data  that  appear  in  maintenance  re¬ 
conditioning  standards. 

9.  Folicy  be  established  pertinent  to  formal  disposition  of 
contractor  "out  of  specification"  material  along  with  appropriate 
modification  to  ASPR  to  prevent  delivery  of  used  or  surplus  material 
without  prior  Government  approval. 

10.  A  study  group  be  established  by  OSD  to  evaluate  the  problems 
attendant  to  adequate  control  of  waivers  of  defects.  This  group  would 
additionally  be  required  to  formulate  working  procedures  that  would 
enable  in-process  identity  of  previous  actions  that  may  have  resulted 

in  waivers  made  on  important  characteristics. 

11.  Programs  be  started  to  promote  development  and  con¬ 
struction  of  adequate  test  equipment  that  also  extends  the  potential 
and  capabilities  of  the  inspector.  The  Defense  budget  should  allow  for 
procurement  of  unique  and,  often,  elaborately  designed  test  gear 
required  in  specialized  applications.  Individual  and  collective 
influences  of  present  equipment,  skills  levels,  mission  constraints, 
etc.  should  be  isolated  and  identified  in  efforts  at  maximizing  the 
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design  configuration.  This  includes  a  requirement  that  the  Services 
and  DSA  place  urgent  priority  on  quantifying  requirements  of  relia¬ 
bility  and  maintainability. 

12.  Policy  be  developed  relating  to  required  training  and 
orientation  in  the  use  and  subsequent  maintenance  of  complex  equip¬ 
ments  so  as  to  lessen  the  likelihood  of  equipment  failures  occurring  as 
a  result  of  abuse  or  misuse, 

13.  Maintenance  Technical  Manuals  be  issued  in  such  a 
manner  as  to  standardize  quality  assurance  requirements  and  pro¬ 
visions.  This  points  the  need  for  a  crisp  format  that  adequately 
expresses  these  requirements  in  common  language.  Also,  the  Services 
should  jointly  investigate  the  problem  of  adequacy  of  engineering 
drawings  considering  maintenance  quality  control  and  accepted 
engineering  design  practices. 

14.  OSD  initiate  action  to  examine  the  existing  deficiency 
reporting  systems  to  determine  wh’ch  of  these  will  provide  the  most 
complete  and  timely  feedback  of  quality  control  deficiency  data. 

This  system  should  then  receive  priority  attention  for  service -wide 
implementation. 

15.  OSD  take  action  to  address  the  attendant  major  problem 
of  coordinating  the  data  feedback  systems  required  by  DODD  3232.  1  - 
Department  of  Defense  Maintenance  Engineering  Program;  DODD 
4100.35  -  Development  of  Integrated  Logistic  Support  for  Systems 
and  Equipments;  DODI  7220.  14  -  Uniform  Cost  Accounting  for  Depot 
Maintenance;  DODI  7730.  25  -  Equipment  Distribution  and  Condition 
(EDAC)  -  Statistical  Reporting  System;  Proposed  DOD  -  Configuration 
Accounting  and  Proposed  DOD  -  Resources  Management;  and  to  place 
these  actions  in  harmony  with  the  intent  of  DODD  5000.  11  -  Data 
Elements  and  Data  Codes  Standardization  Program  and  DODI  5000.  12  - 
Data  Elements  and  Data  Codes  Standardization  Procedures  as  they 
pertain  to  data  element  standardization. 


F.EPORT  OF  PANEL  6 


TI TLE:  Reliability  and  Maintainability  Assessment 

OBJECTIVE: 

To  recommend  actions  essential  to  the  quantitative  assess¬ 
ment  of  materiel  reliability  and  maintainability  during  all 
phases  of  its  life  cycle. 


TOPICS  DISCUSSED: 

1.  Consistent  assessment  criteria  for  Reliability /Main¬ 
tainability 

2.  Reliability /Maintainability  Prediction  Techniques 

3.  Test  Programs  as  an  Input  to  Re  liability /Maintainability 
Assessment 

4.  Field  Data  Feedback 

5.  Reliability  and  Maintainability  Data  Storage  and 
Retrieval 

6.  Research  in  and  Validation  of  Reliability  and  Maintain¬ 
ability  Assessment  Techniques 

7.  Use  of  Reliability  and  Maintainability  Assessment 
Results  by  Management 

8.  Inter  service  Coordination 


29 


GENERAL  BACKGROUND: 


DOD  currently  spends  enormous  sums  on  repair  and  mainte¬ 
nance.  Current  estimates  of  this  run  to  $12  billion  per  year.  Much 
of  this  can  be  saved  by  more  realistic  assessment  of  reliability 
throughout  the  life  cycle  of  equipment. 

Better  assessment  is  needed  to  blend  the  usually  over- 
optimistic  design  and  prototype  probability  prediction  beforehand, 
with  the  more  realistic  production  and  usage  reliability  determination 
afterwards.  Advanced  prediction  techniques  are  necessary  to  avoid 
troubles  experienced  in  previous  systems.  Improved  reporting  is 
necessary  to  pinpoint  the  big  money  users  so  that  appropriate  action 
can  be  taken  on  current  systems,  as  well  as  to  improve  the  prediction 
process  for  future  ones. 

Reliability  and  maintainability  as  addressed  by  Panel  6  consists 
of  a  quantitative  prediction  and/or  measure  of  these  parameters. 

Only  those  activities  associated  with  quantitative  assessment  are 
covered,  thus  eliminating  such  qualitative  assessment  techniques  as 
design  and  program  reviews. 

The  progress  of  the  reliability  and  maintainability  disciplines 
from  art  to  science  is  measured  by  the  ability  to  quantify  those 
parameters.  During  the  past  few  years,  OSD  has  exerted  continuing 
pressure  to  quantify  reliability  and  maintainability.  As  a  result  of 
these  pressures,  virtually  all  new  development  contracts  contain 
quantitative  reliability  and  maintainability  requirements  and  most 
weapon  systems  periodically  report  status  against  these  parameters. 
Thus,  DOD  is  nearing  the  point  where  all  decisions  regarding  develop¬ 
ment  of  new  systems/equipments  are  made  with  the  aid  of  quantitative 
estimates  of  reliability  and  maintainability.  The  next  step  is  to 
extend  this  discipline  through  all  levels  of  management  and  throughout 
the  life  cycle. 

Under  the  guidance  of  OSD,  the  departments  have  made  con¬ 
siderable  progress  in  the  improvement  of  reliability  and  maintainability 
assessment  techniques.  However,  Panel  6  concluded  that  there  were  a 
number  of  serious  weaknesses  in  the  DOD  reliability /maintainability 
assessment  program.  The  progress  being  made  is  far  too  slow  to 
keep  pace  with  management's  need  for  continuing,  consistent,  and 
accurate  knowledge  of  status  against  these  key  parameters. 

Panel  6  concluded  that  DOD  cannot  develop  an  adequate 
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reliability  and  maintainability  assessment  program  without  an 
enhanced  in-house  capability.  The  acquisition  of  this  capability 
should  be  a  major  long-term  goal  of  DOD.  Achievement  of  this  goal 
cannot  be  legislated,  however.  If  the  proper  environment  exists, 
management  at  all  levels  will  conclude  that  additional  in-house 
capability  is  needed  and  will  act  accordingly.  Implementation  of  the 
recommendations  of  the  panel  should  materially  contribute  to  the 
creation  of  that  environment. 


SUMMARY  OF  RECOMMENDATIONS: 


Panel  6  recommends  that: 

1.  OSD  issue  an  instruction  requiring  consistent  assessment 
of  systems /equipments  throughout  their  life  cycle. 

2.  The  Services  give  high  priority  to  the  conduct  of  certain 
reliability  and  maintainability  prediction  research  studies. 

3.  The  Services  intensify  efforts  to  correlate  reliability  and 
maintainability  predictions  to  measured  results  during  the  operational 
phase. 


4.  MIL-STD-785  be  strengthened  by  requiring  additional 
coordination  in  test  planning  between  reliability  and  maintainability 
personnel  and  personnel  responsible  for  test  requirements  prior  to 
the  conduct  of  tests, 

5.  The  Services  increase  use  of  reliability  and  maintainability 
demonstration  requirements  for  more  types  of  equipment,  including 
reprocurements  and  replenishment  spares. 

6.  Each  Service  conduct  an  objective  study  of  its  field  data 
feedback  program  designed  to  re -orient  these  programs  to  provide 
all  data  required  for  effectiveness  assessment. 

7.  OSD  sponsor  a  research  study  of  the  problem  of  accurate 
measurement  of  the  reliability  and  maintainability  of  fielded  systems. 

8.  The  Services  continue  their  use  of  special  reliability  and 
maintainability  monitoring  programs,  especially  during  the 
operational  phase. 
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9.  The  Services  improve  the  reliability  and  maintainability 
coverage  of  their  respective  field  data  feedback  systems, 

10.  DOD  publish  a  handbook  indexing  all  reliability  and  main¬ 
tainability  data  banks  which  would  provide  assessment  personnel 
with  specific  information  on  how  to  obtain  needed  data. 

11.  DOD  establish  a  communication  network  to  provide  for 
rapid  interchange  among  decentralized  data  banks. 

12.  The  Services  intensify  their  research  efforts  in  reliability 
and  maintainability  with  emphasis  on  the  problem  areas  identified  in 
the  Panel  report. 

13.  OSD  establish  a  line  item  in  the  budget  structure  for 
"Research  and  Technique  Development  for  Reliability  and  Maintain¬ 
ability.  " 


14.  The  Services  place  emphasis  on  the  establishment  and  use 
of  assessment  techniques  which  are  specifically  designed  to  aid  the 
executive  in  the  decision-making  process. 

15.  The  Services  undertake  continuing  programs  to  train  all 
levels  of  management  in  reliability  and  maintainability  techniques 
and  terminology. 

16.  DOD  establish  and  chair  a  tri-service  reliability  and 
maintainability  assessment  steering  committee  to  continuously  ex¬ 
change  information,  coordinate  and  advise  OSD  and  the  departments 
regarding  research  needs,  methodology  and  terminology  and  data 
system  design. 
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REPORT  OF  PANEL  7 


TI TLE:  Personnel  and  Training 

OBJECTIVES: 

To  recommend  policies,  procedures  and  actions  essential 
to  insuring  qualified  personnel  for  the  management  and 
implementation  of  quality  and  reliability  operations. 


TOPICS  DISCUSSED: 

1.  The  Diagnosis  of  the  Personnel  and  Training  Problem 

as  it  Relates  to  Quality  and  Reliability  in  the  Department 

of  Defense. 

2.  A  Total  Quality  and  Reliability  Career  Program. 

3.  Acquisition,  Development  and  Retention  of  Quality  and 

Reliability  Skills. 

GENERAL  BACKGROUND: 

In  the  initial  meeting  of  Panel  7  to  discuss  the  area  of 
Personnel  and  Training  in  Quality  and  Reliability  Management,  it 
became  readily  apparent  that  a  great  difference  of  opinion  as  to  the 
role  of  quality  and  reliability  existed  among  the  panel  members;  and 
it  was  felt  these  differences  reflected  the  general  atmosphere  in  which 
quality  and  reliability  activities  function  today.  Little  or  no  progress 
could  be  made  until  there  was  general  agreement  as  to  the  nature  of 
the  role  of  these  skills  across  the  total  logistics  spectrum.  For 
example,  opinion  ranged  from  the  abolishment  of  the  1900  Series  for 
Quality  Control  Specialists  on  the  one  hand,  to  the  opinion  that  this 
particular  area  should  be  the  one  of  major  emphasis.  Convincing 
arguments  were  expressed  supporting  both  views. 

It  is  well  known  that  the  first  step  in  solving  any  problem 
i  s  to  clearly  define  the  problem  and  its  objectives,  It  is  useless  to 
develop  a  cure  until  some  diagnosis  of  the  nature  of  the  existing  illness 
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has  been  made.  In  the  case  of  the  functions  of  quality  and  relia¬ 
bility,  it  was  determined  that  this  diagnosis  must  take  the  form  of 
definitions  of  the  roles  which  these  skills  are  to  play  in  the  total 
product  effectiveness  program. 

Under  existing  conditions,  the  task  of  insuring  ’’qualified 
personnel"  for  the  management  and  implementation  of  quality  and 
reliability  operations  is  an  impossible  one,  since  the  job  for  which 
these  individuals  are  to  be  qualified  has  not  been  clearly  defined. 

An  organization  must  specifically  describe  the  job  to  be  done  in 
terms  of  its  requirements.  One  of  our  major  problems  is  to  attempt 
to  train  or  acquire  talent  only  to  find  it  is  "not  qualified"  for  the  job 
to  be  done  because  we  have  not  described  what  we  want  or  because  it 
is  not  possible  to  match  the  skills  to  the  job. 

Thus,  the  use  of  product  effectiveness  skills  must  be 
carefully  defined:  Is  the  individual  to  be  an  effective  part  of  the 
team  that  contributes  to  decisions  on  design,  on  procurement  and  pro¬ 
duction,  in  the  development  of  storage  and  maintenance  standards  ?  If 
he  is  to  be  totally  effective,  he  must.  If  not,  many  of  the  basic 
premises  upon  which  the  quality  and  reliability  functions  are  based 
are  out  of  date. 

It  became  obvious  in  panel  discussions  that  among  the  many 
changes  taking  place  within  the  Federal  Establishment  is  the  com¬ 
bining  of  a  variety  of  disciplines  into  a  single  program,  which  the 
panel  chose  to  call  the  Product  Effectiveness  Program.  This  com¬ 
bination  of  functions  conceptually  has  not  taken  place  at  the  Depart¬ 
ment  of  Defense  level.  However,  it  has  appeared  in  many  organi¬ 
zations  where  complexity  of  product  has  demanded  it.  Specifically, 
this  combination  of  functions  can  be  seen  in  such  organizations  as 
those  responsible  for  all  Army  missile  systems,  the  Navy  Polaris, 
and  the  Air  Force  Minuteman,  A  recent  study  conducted  by  a  panel 
of  officers  reporting  to  the  Commanding  General  of  the  Army  Materiel 
Command  in  Washington  culminated  in  a  recommendation  that  through¬ 
out  the  entire  Department  of  Army,  this  combining  of  like  functions 
be  effected. 

Therefore,  Panel  7  gradually  evolved  from  "How  do  we 
acquire,  develop  and  retain  quality  and  reliability  skills?"  to  one  of 
"What  is  necessary  in  order  that  an  effective  program  for  the 
acquisition,  development  and  retention  of  quality  and  reliability 
skills  can  be  described?" 
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SUMMARY  OF  RECOMMENDATIONS: 


Panel  7  recommends  that: 

1.  A  clearcut  policy  be  established  with  regard  to  the  roles 
and  responsibilities  of  the  disciplines  of  quality  and  reliability  as 
applied  across  all  elements  of  the  logistics  spectrum,  and  that  these 
disciplines  be  managed  as  a  key  Department  of  Defense  program. 

2.  An  Executive  Agent  for  the  Product  Effectiveness 
Career  Management  Program  be  established  with  a  broad  charter 
to  include  such  things  as  establishment  of  a  career  development 
program  with  mandatory  selection  criteria;  unification  of  existing 
in-service  training;  and  expansion  of  the  curricula  to  meet  defined 
product  effectiveness  program  requirements. 
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REPORT  OF  PANEL  8 


TITLE:  Metrology  and  Calibration  in  Quality  and  Reliability  Operations 

OBJECTIVE: 

To  recommend  action  required  for  the  effective  integration 
and  utilization  of  metrology  and  calibration  in  quality  and 
reliability  operations. 


TOPICS  DISCUSSED: 

1.  Identification  of  New  Measurement  and  Calibration 
Requirements 

2.  a.  Selection  of  Proper  Test  and  Inspection  Equipment 
and  Measurement  Processes 

b.  Adequacy  of  Test  Equipment  and  its  Usage  in  R&D 
Activities 

3.  a.  Promulgation  of  Uniform  Specifications  Controlling 
Contractors  Calibration  Systems 

b.  Inclusion  of  Requirements  for  Calibration  Systems 
in  Small  Business  and  R&D  Contracts 

4.  Provision  of  Metrology  Support  to  Contract  Administration 
Personnel 

6.  Establishment  of  a  Central  Point  of  Contact  within  Office 
Secretary  of  Defense  Responsible  for  Overall  Policy  for 
Metrology  and  Calibration  and  for  Coordination  with  Other 
Government  Agencies 
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GENERAL  BACKGROUND: 


The  deliberations  of  Panel  8  focused  on  consideration  of  the 
degree  of  which  metrology  and  calibration  have  been  woven  into  each 
stage  of  the  life  cycle  of  defense  materiel.  Metrology,  defined  as  the 
science  of  measurement,  concerns  itself  with  the  selection  and 
application  of  test  and  measuring  devices  from  which  quality  decisions 
arc  made;  thus  it  can  be  seen  as  an  element  extremely  essential  to  the 
quality  and  reliability  of  defense  materiel.  The  determination  of  product 
quality  and  the  assessment  of  reliability  levels  depends  principally  upon 
measurement  processes  comprised  of  test  equipment  and  testing  techniques. 

Juct  as  a  day's  production  from  a  manufacturing  process  would 
not  be  accepted  without  evidence  that  the  process  is  in  control,  so  the 
output  of  a  measurement  process  should  not  be  accepted  without  some 
evidence  that  it  is  in  control.  The  process  that  should  be  focused  upon 
is  the  one  which  measures  the  item;  and  this  process  will  include  procedural, 
environmental,  and  other  factors.  Each  link  in  the  chain  from  raw 
material,  to  finished  product,  to  operational  use,  that  involves  measurement, 
should  have  adequate  evidence  that  the  measurement  processes  used  are 
in  control. 


Control  of  these  measurement  processes  is  assured  by  the 
application  of  metrology  to  the  proper  selection  of  equipment  and 
methods,  and  by  the  application  of  calibration  in  maintaining  test 
equipment  to  original  operating  tolerances.  Just  as  the  National 
Bureai  ?  Standards  has  the  responsibility  for  providing  the  basis 
for  a  Iwcional  Measurement  System,  so  the  calibration  laboratories 
of  the  Department  of  Defense  have  the  responsibility  of  insuring  the 
accuracy,  consistency,  and  adequacy  of  that  part  of  the  measurement 
system  that  affects  the  national  defense. 

Panel  8  is  especially  concerned  that  adequate  control  does  not 
exist  at  the  present  time  to  assure  the  proper  selection  of  test  equipment, 
and  that  additional  emphasis  is  required  in  the  application  of  metrology 
and  calibration  at  all  stages  of  the  life  cycle  of  defense  materiel.  While 
it  is  recognized  that  considerable  calibration  effort  is  being  directed 
within  Department  of  Defense  toward  the  operation  and  maintenance 
segment  of  the  life  cycle,  this  effort  is  only  now  beginning  to  prove 
effective  in  controlling  the  quality  of  defense  measurements  at  that 
stage  of  the  cycle. 

Of  even  greater  concern  to  the  Panel  is  the  observation  that  in 
research  and  development  operations,  selection  of  testing  equipment 
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appears  to  be  done  without  the  aid  of  knowledgeable  metrologists  and 
without  the  support  of  adequate  calibration  facilities  to  assure  that 
whatever  equipments  are  selected,  are  maintained  to  their  original 
accuracy  capabilities.  The  determination  of  measurement  problems 
early  in  the  life  cycle  allows  resolution  lead  time  that,  tinder  present 
conditions,  rarely  exists.  By  the  use  of  skilled  metrologists,  avoidance 
of  unnecessary  measurement  problems  critical  to  combat  effectiveness 
can  be  assured. 

The  Panel  noted  particularly  th-.t  in  research  and  development 
operations,  measurement  data  is  frequently  the  product  and  that  "make 
or  break"  procurement  decisions  frequently  are  based  solely  on  these 
data.  Pure  research,  exploratory  development,  advanced  development, 
engineering  development,  and  operation  systems  development  increasingly 
depend  upon  measurement  data.  It  is  essential  that  these  measurement 
data  stem  from  controlled  measurement  processes  which  are  consistent 
and  compatible.  It  is  the  mission  of  metrology  and  calibration  organizations 
and  operations  to  provide  services  which  result  in  control,  conformity, 
continuity,  and  consistency  of  defense  measurement  processes.  More 
careful  application,  support,  and  augmentation  of  these  limited  resources 
are  essential  if  improved  product  quality  and  reliability  are  to  be  expected. 


SUMMARY  OF  RECOMMENDATIONS: 

Panel  8  recommends  that: 

1.  Department  of  Defense  issue  a  directive  or  military  specification 
requiring  systems  development  contractors  or  agencies  to  identify 
measurement  requirements  early  in  the  development  phase. 

2.  Department  of  Defense  (a)  publish  a  technical  guide  for 
Government  and  contractor  engineering  personnel  to  aid  in  the  proper 
selection  of  test  and  inspection  equipment;  (b)  require  R&D  and  design 
engineering  personnel  to  consult  with  metrology  and  calibration 
personnel  in  the  selection  of  test  equipment;  (c)  require  that  test  and 
measuring  instruments  at  R&D  activities  be  subject  to  the  same  quality 
control  measures  as  are  now  required  of  instruments  at  production, 
maintenance  and  field  activities;  (d)  revise  appropriate  military 
specifications  to  require  controls  over  the  selection,  performance 

and  application  of  test  equipment  in  DoD  contracts;  and  (e)  develop  a 
training  course  for  engineering  and  contract  administration  personnel 
in  the  selection  and  application  of  test  and  measuring  equipment. 
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3.  Department  of  Defense  revise  Military  Specification  MIL-C- 
45662A,  Calibration  Systems  Requirements,  and  adopt  it  as  the  standard 
calibration  specification  to  be  inserted  in  Department  of  Defense  and 
National  Aeronautics  and  Space  Administration  contracts,  including  those 
for  small  businesses  and  research  and  development. 

4.  Department  of  Defense  develop  and  promulgate  an  orientation 
program  to  aid  engineering  and  management  personnel  in  recognizing 
the  effect  of  metrology  and  calibration  on  product  quality. 

5.  Department  of  Defense  publish  a  catalog  of  available  metrology 
and  calibration  training  courses,  develop  a  training  course  for  the 
selection  and  application  of  test  and  measuring  processes,  and  ensure 
that  appropriate  engineering  and  contract  administration  personnel  attend 
these  courses. 

6.  Office  Secretary  of  Defense  designate  a  specific  office. 

Office  Assistant  Secretary  of  Defense  (DDR&E,  ASD  (I&L),  or  the 
Department  of  Defense  Quality  and  Reliability  Council)  for  policy 
direction  and  to  serve  as  a  central  point  of  contact  for  metrology  and 
calibration  matters. 
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REPORT  OF  PANEL  9 


TITLE:  Quality  of  Technical  Data 

OBJECTIVE: 

To  improve  the  Department  of  Defense's  capability  to  provide 
data  to  users  that  are  suitable  for  the  intended  purpose,  at 
lowest  practical  cost. 

TOPICS  DISCUSSED: 

1.  Role  of  the  Contractor  in  Data  Quality 

2.  Government  Responsibilities  in  Data  Quality 

3.  Training 

4.  Legibility 

5.  Data  Warranty 

6.  Engineering  Data  Defects  and  "Rights" 

7.  Lack  of  Uniformity  in  Data  Requirements 

8.  Third  Party  Evaluation  of  Data 

9.  Application  of  Zero  Defects  to  Data 

10.  Industry  Viewpoints  on  Data  Adequacy 

11.  Use  of  Data  in  Relationship  to  Quality  Techniques 

12.  Matching  Data  against  Hardware 


41 


GENERAL  BACKGROUND: 


The  importance  of  technical  data  was  recognized  by  the  Secretary 
of  Defense  in  the  early  1960’s  when  management  responsibilities  were 
assigned  in  the  Office  Secretary  of  Defense  staff  and  early  defense 
Directives  issued.  Study  within  the  Department  of  Defense,  assisted  by 
Industry  groups,  and  the  Department  of  Defense  Aberdeen  Technical 
Data  Conference,  demonstrated  the  need  for  larger  permanent  staff  at 
Office  Secretary  of  Defense  and  led  to  the  establishment  in  the  summer 
of  1964  of  the  Office  of  Technical  Data  and  Standardization  Policy  (OTDSP). 
This  Office  is  located  in  Installations  and  Logistics  but  is  responsive  to 
both  logistics  and  research  and  engineering  needs. 

Initial  efforts  of  the  Office  of  Technical  Data  and  Standardization 
Policy,  as  they  pertain  to  quality,  were  directed  principally  at  placing 
a  discipline  on  the  ordering  of  data  from  contractors  to  obtain  better 
definition  of  data  requirements  and  prevent  ordering  of  data  in  excess 
of  needs.  All  data  requirements  (DoD  Directive  5010.12)  must  be  placed 
in  on  single  form  in  all  contracts,  the  DD  Form  1423,  and  must  be 
selected  from  a  predetermined  list  of  data  requirements  (Authorized 
Data  Lists).  Recently,  the  Office  of  Technical  Data  and  Standardization 
Policy,  recognizing  the  importance  of  data  quality,  established  an 
interdepartmental  committee  to  develop  procedures  to  strengthen  and 
make  more  uniform  the  data  quality  assurance  practices  of  the  Military 
Services. 

By  data  quality,  we  mean  that  data  delivered  has  utility  for  the 
purpose.  Utility  is  affected  by  the  formulation  of  the  data  requirement, 
the  communication  of  the  requirement  to  the  contractor,  the  method  of 
generation  the  inspection  and  acceptance,  and  the  handling  of  data 
before  it  is  placed  in  the  hands  of  the  ultimate  user.  For  data  to 
have  utility  it  must  have  the  proper  content,  be  in  a  readily  usable  form 
(format  considerations)  and  be  legible. 

Technical  data  permeates  every  function  and  discipline  in  the 
Department  of  Defense.  Because  of  the  extremely  broad  range  of  kinds 
of  data,  the  complexity  of  the  subject  in  general,  and  the  number  of 
actions  involved  that  have  an  effect  on  data  quality,  Panel  9  found  it 
necessary  to  concentrate  on  selected  aspects  of  data,  and  to  exclude 
others,  even  though  they  may  have  significant  impact  on  data  quality. 

For  the  most  part,  we  have  emphasized  the  contractual  aspects  of 
engineering  drawing  acquisition,  although  peripheral  subjects  are 
included.  The  majority  of  the  recommendations  made  apply  to  other 
kinds  of  darn  as  well  as  drawings. 
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SUMMARY  OF  RECOMMENDATIONS: 


Panel  9  recommends  that: 

1.  Department  of  Defense  develop  and  specify  the  minimum 
elements  of  an  acceptable  contractors  data  quality  program  in  a  military 
specification,  preferably,  MIL-Q-9858. 

2.  Department  of  Defense  include  a  detailed  statement  of  the 
Government's  responsibilities  for  data  quality  in  Department  of  Defense 
Directive  5010.12. 

3.  Department  of  Defense  develop  a  Department  of  Defense 
handbook  outlining  procedures  for  reviewing  adequacy  of  contractors 
technical  data  control  system. 

4.  Department  of  Defense  revise  provisioning  regulations  to 
require  that  Supply  Support  Requests  identify  the  method  of  procurement, 
justify  the  method,  and  provide  data  suitable  to  support  it. 

5.  Department  of  Defense  develop  a  data  training  syllabus  for 
data  inspectors. 

6.  The  Armed  Services  Procurement  Regulation  Committee 
favorably  consider  a  data  warranty  clause  developed  by  Air  Force/ 
Industry. 

7.  The  Services  use  photorestoration  processes  for  presently 
illegible  drawings  rather  than  redraw. 

8.  The  Services  continue  emphasis  on  Government  inspection 
for  legibility  using  technically  trained  personnel. 

9.  Department  of  Defense  continue  emphasis  on  achieving 
uniformity  in  military  drafting  practices. 

10.  Department  of  Defense  support  and  provide  leadership  to 
National  Standardization  efforts  in  drafting  practices. 

11.  The  Services  use  ’’third  party"  contractors  to  evaluate  data 
only  on  a  case-by-case  basis.  Such  contractors  should  not  be  used  for 
evaluation  of  technical  content  except  under  unusual  circumstances. 

12.  The  Services  include  data  considerations  in  present  Zero 
Defects  programs. 
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13.  Department  of  Defense  consider  objectively  Industry  comments 
on  data  quality  that  have  been  received. 

14.  Department  of  Defense  expedite  a  project  to  expand  the 
techniques  of  MIL-STD-789  (ASG)  for  Procurement  Method  Coding  in  a 
Department  of  Defense-wide  document.  Special  quality  assurance 
consideration  should  be  given  to  items  designated  for  competitive 
procurement. 

15.  The  Services  ensure  that  contractors  data  quality  programs 
contain  provisions  to  match  data  against  hardware. 

16.  The  Services  review  present  contracts  to  apply  present  "rights” 
provisions  of  Armed  Services  Procurement  when  economically  practical. 

17.  The  Services  validate  drawings  by  building,  testing,  and 
obtaining  feedback  from  the  maximum  number  of  items  practical. 
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KEYNOTE  ADDRESS 
BY 

MR.  GEORGE  E.  FOUCH 
DEPUTY  ASSISTANT  SECRETARY 
(EQUIPMENT  MAINTENANCE  &  READINESS) 

OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE  (I&L) 


Mr.  Chairman,  Distinguished  Guests,  and  Panelists: 

On  behalf  of  the  Director  of  Defense  Research  and  Engineering 
and  the  Assistant  Secretary  of  Defense  (I&L),  it  is  my  great  pleasure 
to  welcome  you  to  this  Conference  on  Quality  and  Reliability  Manage¬ 
ment.  The  degree  of  success  of  this  Conference  has  been  largely  pre¬ 
determined  by  the  work  that  the  chairmen  and  panelists  have  accom¬ 
plished  over  the  past  six  or  eight  months.  Neither  extended  comment 
nor  exhortation  by  me  can  substantially  add  to  or  subtract  from  the 
quality  of  that  work.  Thus,  my  comments  will  not  be  extended. 

This  is  the  sixth  DoD-wide  conference  that  the  Office  of  the 
Secretary  of  Defense  has  sponsored  since  February  1962  in  the  field 
of  materiel  and  management.  These  conferences  have  led  to  important 
improvements  in  management  practices  in  the  Department  of  Defense, 
They  have  led  also,  I  might  add,  to  improvements  in  the  methods  by 
which  we  conduct  conferences.  In  former  times,  meetings  of  this 
kind  have  led  frequently  to  severe  frustrations;  participants  did  not 
have  time  to  explore  fully  their  mutual  problems  in  a  period  of  three 
or  four  days.  This  Conference  has  been  so  planned  that  problems 
have  been  explored  in  advance  of  this  Annapolis  meeting.  During  the 
next  few  days,  therefore,  you  will  be  primarily  concerned  with 
polishing  the  ideas  and  recommendations  you  have  already  formulated 
and  exchanging  views  among  panels. 

We  are  most  hopeful  that  this  Conference  will  play  the  same 
role  in  Quality  and  Reliability  Management  that  previous  conferences 
have  played  in  various  other  fields  of  management.  The  Defense 
Contract  Administration  Service,  for  example,  has  its  roots  in  the 
procurement  management  conference  at  Williamsburg  in  1962. 
Important  advances  have  been  made  in  technical  data  management 
as  a  direct  result  of  the  conference  at  Aberdeen  in  1964.  It  is  quite 
apparent  that  the  highest  levels  of  management  of  the  DoD  are 
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receptive  to  the  recommendations  that  derive  from  conferences  of  this 
kind.  You  can  be  sure  that  your  voice  will  be  heard. 

The  objectives  of  this  Conference  have  been  spelled  out  quite 
clearly  by  the  OSD-Army-Navy-Air  Force -DSA-NASA  Steering 
Committee.  It  may  be  useful  to  repeat  these  objectives  so  that  all 
of  us  keep  in  mind  exactly  why  we  are  here.  They  are  fourfold: 

1.  To  analyze  the  principal  quality  and  reliability  problems 
that  confront  the  DoD; 

2.  Assess  pertinent  current  efforts  and  proposed  plans  which 
are  aimed  at  solving  these  problems; 

3.  Exchange  ideas  on  solution  of  these  problems;  and 

4.  Recommend  to  executive  levels  of  the  OSD,  the  Military 
Departments,  and  DSA 

a.  appropriate  and  immediate  actions  to  resolve  specific 
quality  and  reliability  problems,  and 

b.  long-range  plans  and  programs  for  improving  quality 
and  reliability  assurance  efforts. 

As  stated,  these  objectives  are  phrased  in  the  somewhat  stilted 
bureaucratese  in  which  most  of  us  indulge.  Clearly,  the  real  reason 
we  are  here  is  to  promote,  through  improved  quality  and  reliability 
management  practices,  the  well-being  of  our  country.  Some  months 
ago,  Deputy  Secretary  Vance  expressed  this  thought  succinctly  when 
he  said: 

"The  military  security  of  the  United  States  requires  that  we 
remain  ahead  of  the  technological  competition  for  better 
and  more  reliable  weapons  systems  --  and,  above  all,  that 
we  excel  in  managing  the  resources  entrusted  to  us  for 
developing  and  producing  the  vast  amounts  of  materiel 
required  to  equip  and  maintain  the  forces  needed  for  our 
military  security.  " 

It  need  hardly  be  said  that  few  areas  of  management  have  a  greater 
impact  on  the  proper  utilization  of  resources  than  quality  and 
reliability. 
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This  Conference  has  been  convened  at  a  particularly  timely 
moment.  As  a  nation  we  are  all  concerned  with  providing  our  men  in 
Viet  Nam  with  the  supplies  and  equipment  they  need  --  promptly  and 
efficiently.  Sometime  ago,  General  Wheeler  described  Viet  Nam  as 
an  event  in  "the  longest  War"  -  the  war  that  has  flamed  and  flickered 
sii.ce  1945.  Locales  have  constantly  changed  -  -  Berlin,  Korea,  the 
Formosan  Straits,  Cuba,  and  now  Viet  Nam.  But  the  bed-rock 
situation  has  not  changed  --  we  have  been  repeatedly  probed  and 
challenged.  This  is  a  probing  of  our  determination  to  protect  the 
cause  of  human  freedom.  But,  as  Secretary  Ignatius  recently 
commented: 

"This  determination  is  not  solely  a  matter  of  military  action. 

Not  all  of  us  can  be  in  the  jungles  and  swamps  of  Viet  Nam. 

No  less  essential  is  determination  in  the  everyday  jobs  of 
fabricating,  maintaining  and  furnishing  supplies  and  equip¬ 
ment  to  those  who  bear  the  heat  and  burden  of  the  day.  We 
are  determined  that  the  job  will  be  done  effectively." 

Again,  it  is  quite  apparent  that  quality  and  reliability  management  play 
a  central  role  in  providing  the  effective  support  to  Viet  Nam  to  which 
Secretary  Ignatius  made  reference.  This  is  especially  true  with 
respect  to  the  hour-to-hour,  day-to-day  effectiveness  with  which  we 
examine  and  test  the  supplies  and  equipment  flowing  from  industrial 
and  military  industrial  activities  to  the  operating  commands. 

I  said  that  this  meeting  has  been  convened  at  a  particularly 
timely  moment.  Apart  from  Viet  Nam,  we  as  a  nation  have  become 
increasingly  concerned  with  the  quality  and  reliability  of  the  products 
we  use  in  our  everyday  living.  This  concern  is  quite  apparent  from 
reading  the  daily  news  reports,  not  to  mention  irate  letters  to  editors 
regarding  some  less -than- satisfactory  consumer  products  and  services. 
This  Conference  is,  of  course,  addressed  to  problems  of  the  DoD.  It 
may  v/ell  be,  however,  the  ideas  and  suggestions  that  stem  from  this 
Conference  will  make  an  important  contribution  to  the  continued  progress 
of  our  national  economy.  In  my  work  in  the  OSD  I  have  been  encouraged 
and  inspired  by  the  knowledge  that  techniques  and  concepts  developed 
originally  for  defense  purposes  have  contributed  importantly  to  social 
and  industrial  progress.  This  has  been  notably  true  with  respect  to 
theory  and  practices  related  to  quality  and  reliability.  Accordingly, 
as  panelists  and  participants  in  this  Conference,  you  are  privileged 
to  contribute,  directly  and  indirectly,  to  improving  the  well-b^ing  of 
the  American  people. 


47 


Let  me  say  a  word  about  the  climate  of  this  Conference.  First 
and  foremost  --  from  the  very  beginning  we  have  encouraged  a  free 
expression  of  opinion.  We  are  meeting  here  as  free  spirits,  working 
in  a  climate  of  free  inquiry.  I  hope  that  this  climate  of  free  expression, 
as  well  as  tolerance  of  each  other's  points  of  view,  will  be  maintained 
during  the  next  few  days.  There  is  no  reason  why  we  cannot  be  com¬ 
pletely  frank  in  our  opinions.  This  is  particularly  true  at  a  meeting 
of  this  type;  we  are  here  as  a  family  of  people  examining  how  we  our¬ 
selves  conduct  our  affairs.  This  is  an  in-house  self  appraisal.  Let 
us,  therefore,  be  forthright  and  tolerant.  If  any  single  panelist 
strongly  dissents  from  the  consensus  of  his  colleagues,  he  is  invited 
to  submit  a  separate  statement  of  his  views.  These  minority  state¬ 
ments  will  be  carefully  studied  as  Lhe  Office  of  the  Secretary  of  Defense, 
the  Military  Departments,  and  the  various  Agencies  review  and  imple¬ 
ment  the  Conference  report.  Understandably,  we  hope  for  consensus. 
But  we  do  not  want  consensus  at  the  price  of  full  ventilation  of  opinion. 
We  hope  that  this  Conference  will  generate  practical  and  useful 
recommendations  for  the  continued  improvement  of  quality  and 
reliability  management  practices. 

Possibly  a  keynote  speaker  may  be  forgiven  if  he  takes  a  few 
moments  from  his  allotted  time  to  explore  some  subjects  that 
particularly  concern  him.  First,  we  all  know  that  from  time  to  time 
there  occurs  in  the  public  press  or  elsewhere  a  rash  of  questioning 
statements  regarding  the  quality  and  reliability  of  military  or  space 
supplies  and  equipment.  These  statements,  including  some  that  have 
originated  from  my  office,  have  served  a  useful  purpose.  They 
negate  any  tendency  towards  complacency.  But,  they  pertain  only 
to  limited  segments  of  materiel.  Such  reports  do  not  reflect  the 
big  picture.  We  lack  a  device  for  surveying  this  larger  scene.  We 
need  a  continuing  and  sensitive  system  for  measuring  the  quality  and 
reliability  of  the  products  we  acquire  from  industry  to  produce, 
store  and  maintain  in  in-house  establishments.  This  is  not  to 
suggest  that  the  departments  and  various  agencies  do  not  have  up- 
to-date  quality  and  reliability  information  on  selected  products.  But, 
comprehensive  information  is  not  readily  available  to  upper  level 
management  throughout  the  Department  of  Defense,  In  recent  years 
I  have  chaired  semi-annual  meetings  at  which  representatives  of  the 
Departments  and  DSA  have  met  and  exchanged  the  quality  and  re¬ 
liability  information  they  have  acquired.  The  fact  remains,  however, 
that  we  lack  a  continuing  program  to  keep  track  of  trends  in  quality 
and  reliability.  In  former  times  we  could  justify  this  deficiency  by 
reason  of  lack  of  an  adequate  data  collecting  and  analyzing  capability. 

This  kind  of  justification  has  less  substance  today  in  this  era  of 
computerization. 
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Secondly,  the  Department  of  Defense  expends  considerable 
resources  in  making  sure  that  products  and  services  are  useable 
and  reliable.  We  have  sizeable  programs  aimed  at  combating  trends 
towards  defectiveness  in  production  and  maintenance.  We  don't 
know,  however,  what  defectiveness  actually  costs.  The  joint  DoD- 
NASA  Quality  and  Reliability  Assurance  Committee  recently  recom¬ 
mended  that  a  study  be  made  to  assess  the  economic  dimensions  of 
defectiveness.  Such  an  assessment  would  be  highly  useful  not  only 
to  NASA  and  to  the  DoD,  but  to  the  Government  generally. 

Thirdly,  I  have  been  a  little  concerned  that  we  may  be  moving 
in  the  direction  of  a  paper-heavy  bureaucracy  in  DoD  quality 
assurance.  The  establishment  of  the  Defense  Contract  Administration 
Service  has  improved  the  ability  of  the  DoD  to  conduct  its  business  with 
industrv  more  efficiently.  We  have  reason  to  be  concerned,  however, 
that  Government  quality  assurance  might  become  entangled  in  a  web 
of  well-intentioned  but  excessive  guidance  from  technical  and  pro¬ 
curing  agencies.  It  would  appear  to  me  that  the  first  line  of  defense 
against  inferior  quality  and  reliability  is  the  contract.  If  the  contract 
and  the  related  specifications  are  not  clear,  complete  and  explicit, 
we  can  hardly  expect  the  contractors  to  give  us  what  we  want.  No 
amount  of  quality  assurance  by  the  Government  will  compensate  for 
deficient  contract  packages.  Equally  important,  the  Government 
quality  assurance  activities  must  have  strong  technical  competence 
and  flexibility  in  utilizing  that  competence  when  servicing  the  Army, 
Navy,  Air  Force,  and  DSA.  I  will  not  expand  on  this  --  I  am  simply 
mentioning  this  subject  with  the  thought  --  paper  work  should  not 
supersede  attention  to  the  essentials  of  inspection,  process  control 
and  testing. 

Fourthly,  as  I  have  indicated  on  other  occasions,  I  am  con¬ 
vinced  that  something  has  to  be  done  to  interrelate  and  possibly 
integrate  the  many  management  systems  and  movements  that  keep 
cropping  up.  These  movements  are  responses  to  real  problems. 

But  I  am  apprehensive  lest  people  in  Government  and  industry 
become  disenchanted  with  the  new  management  techniques  for 
reasons  of  non-integration.  I  am  making  reference,  for  example, 
to  such  functions  as  configuration  control,  value  engineering,  PERT, 
Zero  Defects,  standardization,  cataloging,  total  package  procurement, 
specification  management,  total  logistics  support,  quality  control, 
reliability  and  maintainability.  And  this  does  not  exhaust  the  litany 
of  new  systems,  methods,  and  movements  that  we  all  know  about. 

I  repeat  that  each  of  these  represents  a  response  to  a  real  problem. 

In  that  respect  they  contribute  to  management  effectiveness.  How¬ 
ever,  these  various  techniques  appear,  at  times,  to  be  ends  in 
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themselves,  isolated  from  each  other  as  well  as  from  the  central 
objectives  of  management.  Possibly  this  Conference  might  come  up 
with  ideas  regarding  how  we  can  capitalize  to  the  maximum  on  the 
intrinsic  dynamism  of  these  movements  without  perpetuating  com¬ 
peting  sects  and  cults. 

Fifth,  I  have  shared  the  apprehension  of  other  persons  in 
the  DoD  that  we  might  inadvertently  sacrifice  quality  for  competition. 

In  a  letter  to  tne  Secretaries  of  the  Army,  Navy  and  Air  Force,  and 
Director  of  the  DSA,  Secretary  McNamara  recently  said  that  he  is 
"proud  of  our  continued  high  record  of  price  competition,"  but  he 
"does  not  want  this  record  to  be  at  the  expense  of  quality  of  product 
or  quality  of  performance."  Apart  from  use  of  preaward  surveys 
and  other  techniques  to  prevent  the  selection  of  contractors  with 
marginal  capability,  we  can  protect  quality  by  improving  specifications, 
by  enforcing  contracts  and  specifications  firmly  but  fairly,  and  by 
utilizing  the  services  of  our  quality  assurance  personnel  more 
effectively  where  they  can  do  the  most  good;  namely,  at  the  plant 
level.  There  is  need,  as  Secretary  McNamara  has  indicated,  for 
continuing  attention  to  the  problem  of  selecting  competent  sources. 
There  is  also  need  for  equally  strong  attention  to  the  problem  of 
enforcing  provisions  of  contracts  through  more  effective  utilization 
of  quality  assurance  personnel. 

Lastly,  I  want  to  mention  the  need  for  attention  to  the  human 
dimensions  of  reliability.  The  Zero  Defects  Program  has  demonstrated 
the  widespread  concern  of  people  in  Government  and  industry  for 
personal  identification  and  recognition.  People  resent  being  de¬ 
personalized.  Yes,  they  want,  first  and  foremost,  adequate  compen¬ 
sation  for  their  work.  But  they  also  want  something  more.  They  want 
to  be  recognized  as  individuals.  They  want  to  feel  that  they  —  and 
their  ideas  --  play  a  useful  and  recognizable  role  --  however  modest  -- 
in  the  organizations  of  which  they  are  a  part.  The  human  aspect  of 
reliability  and  quality  management  is  a  vast,  complex  and  somewhat 
uncharted  subject.  Certainly  it  is  a  subject  beyond  my  ability  to 
treat  adequately  during  these  brief  comments.  It  seems  to  me,  how¬ 
ever,  that  in  various  ways  --  through  training,  through  improved 
personnel  practices,  through  award  and  recognition  programs  --  we 
could  do  a  lot  more  than  we  are  doing  on  the  person-to-person  level 
to  improve  quality  and  reliability. 

I  have  imposed  on  your  good  will  for  the  last  three  or  four 
minutes  to  mention  some  of  the  things  that  personally  concern  me.  I 
am  not  suggesting,  however,  that  these  problems  have  greater 
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urgency  than  the  many  subjects  you  have  discussed  at  your  meetings 
prior  to  coming  to  Annapolis. 

Two  days  from  now  --on  Thursday  --  you  will  reconvene  in 
this  room  to  present  your  reports  and  recommendations  to  an 
important  cross  section  of  executives  of  the  DoD.  Subsequently, 
these  recommendations  will  be  incorporated  into  a  conference  report. 
The  published  report  can  be  expected  to  have  a  long-term  effect  -- 
hopefully  a  favorable  effect  --on  quality  and  reliability  management. 
We  expect  that  your  comments  will  be  expressed  in  clear,  forthright 
language  and  that  your  recommendations  --  whether  of  the  short  term 
or  long-term  variety  --  are  reasonably  amenable  to  implementation. 
The  adoption  and  implementation  of  your  recommendations  will  be 
expedited  considerably  by  the  clarity  and  practicability  of  what  you 
have  to  say. 

In  the  final  analysis,  however,  your  recommendations  will  be 
judged  by  one  criterion;  namely,  their  utility  in  providing  our  armed 
services  with  supplies  and  equipment  at  minimum  cost.  It  is  well  to 
recall  the  commission  originally  given  to  Secretary  McNamara  by 
President  Kennedy,  and  reaffirmed  by  President  Johnson.  Secretary 
McNamara  was  directed  to: 

1.  Develop  the  force  structure  necessary  to  our  military 
requirements  without  regard  to  arbitrary  budget  ceilings. 

2.  Procure  and  operate  this  force  at  the  lowest  possible  cost. 

We  are  here  to  rethink  policies  and  procedures  in  quality  and 
reliability  management  with  special  attention  to  the  second  sentence 
of  the  Kennedy- Johnson  mandate,  "Procure  and  operate  this  force  at 
the  lowest  possible  cost.” 

May  I  thank  the  Conference  Director,  the  Steering  Group  and 
the  Chairmen  and  panelists  for  their  excellent  cooperation  and 
initiative  in  planning  and  preparing  for  this  conference.  I  hope  these 
three  days  will  be  rewarding  to  you  as  participants  and  to  the  people 
we  ultimately  serve  --  the  armed  forces  on  the  frontiers  of  defense 
overseas  and  our  neighbors  and  the  American  public  here  at  home. 
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Much  of  the  emphasis  given  to  Reliability  and  Quality  Assurance 
in  the  past  ten  years  stemmed  from  our  Defense  missile  programs  and, 
more  recently,  from  our  space  efforts.  However,  let  me  assure  you 
that  the  importance  of  Reliability,  Maintainability  and  Quality  in  the 
many  other  Defense  systems  and  equipment  is  at  times  even  more 
critical  in  view  of  their  required  effectiveness  and  the  many  lives  that 
may  depend  on  these  factors. 

You  know  that  one  of  the  prime  objectives  of  the  DoD  is  the 
acquisition  and  deployment  of  adequate  weapon  systems  in  a  proper 
time  frame  at  reasonable  costs.  By  "adequate"  we  mean  responsive  to 
the  specifications  which  in  turn  must  reflect  our  operational  needs. 

And  in  assuring  reasonable  costs  we  must  consider  cost  of  ownership 
over  the  system  life. 

Achieving  these  objectives  and  policies  in  the  current  environment 
demands  techniques  for  improved  program  management  and  technical 
control.  Very  briefly  our  situation  is  one  of  (a)  complex,  high  performance, 
and  quite  expensive  systems;  (b)  these  are  resulting  in  increasingly 
complex  contractual  and  management  arrangements  and;  (c)  we  are 
experiencing  costly  maintenance  and  logistic  support.  The  quantitative 
approach  and  analytical  techniques  for  reliability  and  maintainability, 
though  incomplete  in  some  areas,  have  proved  themselves  as  powerful 
tools  for  design  and  management  decisions. 

I  wish  to  suggest  that  there  are  several  broad  challenges  to 
Reliability  and  Quality  Assurance  today.  We  need  to  continue  the 
improvement  of  analytical  and  audit  techniques  for  Reliability  and 
Maintainability;  and,  these  improved  techniques  need  to  be  fully 
integrated  into  the  design  and  engineering,  into  configuration  management 
and  change  control,  and  into  the  overall  program  management  process. 
Refined  and  improved  quality  assurance  quantitative  techniques  are 
needed  especially  in  the  extreme  conditions  where  normal  statistical 
techniques  become  unusable.  Also  we  need  better  control  in  the 
operational  portion  of  the  life  cycle;  that  is,  in  maintenance,  supply, 
and  storage. 
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I  would  like  to  offer  two  suggestions  in  the  form  of  specific 
challenges  to  reliability  and  quality  assurance: 

First  -  Full  use  is  not  being  yet  made  of  recently  developed 
quantitative  methods.  This  may  be  due  to  the  fact  that  little  time  has 
elapsed  since  changed  instructions  were  issued.  Education  is  needed  in 
the  use  of  these  improved  analytical  and  audit  techniques  for  reliability, 
maintainability  and  quality  assurance.  The  efforts  expended  in  their 
development  are  wasted  unless  they  are  put  to  use  by  all  levels  of  line 
management  in  program  decisions,  by  engineers  in  their  designs,  by 
procurement  officers  invoking  these,  contractually,  by  maintenance  and 
by  other  logistics  support  people.  You,  who  are  the  specialists,  and 
who  form  the  really  significant  competence  in  these  techniques  and  methods 
must  increase  your  efforts  to  educate  the  line  managers,  the  designers, 
procurement  officers  and  the  operational  support  types.  You  must  assist 
the  various  groups  in  the  use  and  continued  development  of  advanced 
techniques  (i.  e. ,  predictions,  apportionment,  trade-off  and  system 
analysis,  assessment  of  reliability  and  maintainability,  design  review,  etc.). 

A  Second  Suggestion  -  recognizes  the  continued  need  for 
refinement  and  that  gaps  still  exist  in  the  technology.  These  are  the 
areas  to  which  we  must  direct  our  studies  and  research.  I  have  in  mind 
the  ever  present  problem  of  adequate  testing  within  the  constraints  of 
schedule  and  funds.  This  is  extremely  critical  where  systems  are 
designed  for  very  long  MTBF  or  Mean  Life  (satellites,  communications 
switching  systems,  radio  receiving  equipment,  etc.).  Schedules  and 
costs  do  not  permit  long  term  reliability  testing  using  conventional 
test  plans  or  sometimes,  even  the  use  of  truncated  test  plans.  Similarly, 
in  the  parts  area  there  are  some  extremely  long  life  electronic  devices, 
including,  some  of  the  recently  developed  microelectronics  integrated 
circuit  configurations.  We  can  no  longer  afford  the  long  term,  mass 
testing  which  has  been  developing  in  the  past  few  years.  We  need  newer 
test  and  inspection  methods  which  will  permit  assessment  of  the  life 
characteristics  of  these  devices  from  short  term  tests  or  inspections. 

Going  back  to  systems,  we  are  developing  expensive,  complex, 
expendable  systems  which  are  not  susceptible  to  reliability  assessment 
using  standard  statistical  test  plans.  Assessment  with  limited  test 
data  is  a  real  problem  needing  study  and  probably,  innovation. 

Techniques  such  as  accelerated  testing,  design  analysis,  and  physics 
of  failure  correlated  to  failure -rate  at  the  equipment  and  systems  level 
have  not  been  fully  exploited. 
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With  respect  to  testing,  particularly  in  the  equipment  and  systems 
areas,  the  concept  of  integrated  test  plans  should  be  employed.  Further, 
optimization  of  these  plans  based  on  previous  reliability  information  of 
component  parts  could  result  in  substantial  savings  in  terms  of  cost, 
time  and  facilities.  And  I  might  add,  with  more  meaningful  results. 

In  the  prediction  area,  reliability  prediction  in  the  mechanical 
and  hydraulic  areas  needs  emphasis. 

Of  course,  design  techniques  and  materials  which  will  circumvent 
principal  failure  mechanisms  or  shield  devices  or  even  full  equipments, 
from  damaging  environments  are  areas  of  continued  research;  these  can 
have  substantial  effect  on  your  areas  of  responsibility. 

One  last  comment  on  techniques  -  we  must  be  certain  to  keep 
these  recently  developed  techniques  current  and  compatible  with  other 
advances  in  defense  systems  technology;  for  example,  reliability 
prediction  techniques,  process  control  and  quality  assurance  techniques 
applicable  to  microelectronics  and  solid  state  designs. 

I  have  gone  to  a  degree  of  detail  for  several  reasons.  Techniques 
for  predicting,  apportioning  and  measuring  reliability  and  maintainability 
give  us  excellent  tools  to  increase  the  effectiveness  of  our  weapons 
systems.  But  these  same  techniques  also  result  in  the  achievement  of 
increased  management  effectiveness.  For  example  they  serve  as  tools 
for  the  following; 

1.  to  evaluate  feasibility  and  alternative  designs  and  to 
establish  technically  realistic  contract  requirements; 

2.  to  perform  more  effective  trade-offs,  system  analysis, 
system  effectiveness  estimates  and  better  Contract  Definition; 

3.  to  improve  evaluation  of  configuration  changes,  value 
engineering,  or  cost- reduction  changes; 

4.  to  form  a  better  basis  for  estimating  force  structures; 

5.  to  permit  to  a  greater  degree  the  use  of  Life  Cycle 
Costing  as  a  basis  for  contract  award;  and 

6.  to  plan  better  maintenance  and  integrated  logistic 

support. 
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So  you  see  that  reliability  and  maintainability  techniques  with 
attendant  quality  assurance  measures  result  in  effective  tools  for 
management  in  many  areas  of  operation. 

While  we  have  highlighted  some  of  the  more  important  techniques 
developed  to  specify,  predict  and  measure  reliability  and  maintainability, 
we  must  not  forget  the  relationship  of  a  critical  area  -  Production  and 
Maintenance.  Workmanship  defects  can  be  a  death  knell  denying  the 
achievement  of  reliability  and  maintainability  which  is  inherent  in  the 
design.  Industry  and  Government  must  rely  very  heavily  on  the  Quality 
Control/ Quality  Assurance  functions  to  see  that  inherent  high- reliability 
designs  do  in  fact  become  high- reliability  hardware  and  are  maintained 
at  these  levels. 

In  conclusion  I  wish  to  compliment  all  of  you  who  participated  in 
this  conference.  I  understand  that  high  morale  and  interest  have  been 
evident  and  most  impressive.  Your  conference  objectives  must  have 
been  achieved.  Certainly  there  must  have  been  a  great  deal  gained  from 
the  interchange  of  information  between  participants  from  the  Military 
Departments,  Defense  and  other  Government  Agencies.  I'm  sure  that 
a  detailed  study  of  your  recommendations  will  result  in  improvement  and 
innovation.  Secretary  Ignatius  and  I  both  wish  to  thank  you  for  your 
efforts.  We  urge  you  to  continue  these  efforts  in  this  area  so  vital 
to  our  fighting  forces.  I  know  that  you  will;  and  that,  throughout  the 
world  men  who  would  otherwise  die,  will  live  as  a  direct  result  of  your 
efforts. 


